Hi Lee, sorry for long delay in my response, but I wanted to fully test it and thanks to your draft I was able to decrypt my encrypted volume. Your procedure is well described, so I just collected all steps in one place and wanted to share in this conversation: Jan Wasilewski [image: Załączniki]11:04 (1 minutę temu) do mnie **Description:** As an admin, you would like to decrypt the volume, which is attached to compute node and check, that your barbican secret key is correct(i.e. customer is saying, that the barbican secret key doesn't work). This procedure describes, how you can simply test it. **Starting point:** Volume is encrypted and attached to an instance(instance has to be shutdown to make qemu commands operational). Our volume id is: ca8da832-a88d-4f91-ab2d-2bd3efbca4a3 **Procedure:** Log in to a compute node that is hosting your instance. List volumes attached to your instance: ``` [TEST]root@comp-09:/home/jwasilewski# virsh domblklist ec9081e4-e1e4-40a2-bf8c-c87c14b79d5a Target Source ------------------------------------------------ vda /dev/dm-29 vdb /dev/disk/by-id/wwn-0x6e00084100ee7e7e7fe79b5900003a89 ``` In our case vdb volume is an encrypted one. We can check it by qemu-img command: ``` [TEST]root@comp-09:/home/jwasilewski# qemu-img info /dev/disk/by-id/wwn-0x6e00084100ee7e7e7fe79b5900003a89 image: /dev/disk/by-id/wwn-0x6e00084100ee7e7e7fe79b5900003a89 file format: luks virtual size: 20G (21472739328 bytes) disk size: 0 encrypted: yes Format specific information: ivgen alg: plain64 hash alg: sha256 cipher alg: aes-256 uuid: 009f60f7-e871-4eac-88da-b274e80eb247 cipher mode: xts slots: [0]: active: true iters: 900838 key offset: 4096 stripes: 4000 [1]: active: false key offset: 262144 [2]: active: false key offset: 520192 [3]: active: false key offset: 778240 [4]: active: false key offset: 1036288 [5]: active: false key offset: 1294336 [6]: active: false key offset: 1552384 [7]: active: false key offset: 1810432 payload offset: 2097152 master key iters: 56302 ``` We would like to decrypt the volume. We need to retrieve symmetric key which is allocated to this volume from barbican. We need to find a secret store associated with our volume, so we have to login to OpenStack database and execute: ``` mysql> select * from volumes where id = 'ca8da832-a88d-4f91-ab2d-2bd3efbca4a3'\G *************************** 1. row *************************** created_at: 2021-02-12 13:41:40 updated_at: 2021-02-17 12:33:34 deleted_at: NULL deleted: 0 id: ca8da832-a88d-4f91-ab2d-2bd3efbca4a3 ec2_id: NULL user_id: wfoij24f0sdfs0934nkl project_id: 234sfds90klfgd093n host: cinder-01@huawei_backend#StoragePool001 size: 20 availability_zone: nova status: in-use attach_status: attached scheduled_at: 2021-02-12 13:41:40 launched_at: 2021-02-12 13:41:42 terminated_at: NULL display_name: encrypted-volume display_description: provider_location: {"huawei_sn": "2102352VVA10L2000001", "huawei_lun_id": "14985", "huawei_lun_wwn": "6e00084100ee7e7e7fe79b5900003a89"} provider_auth: NULL snapshot_id: NULL volume_type_id: 3129bdc2-6162-4729-9eab-d0c97db2335a source_volid: NULL bootable: 0 provider_geometry: NULL _name_id: NULL encryption_key_id: b13d2017-e3e5-4f5f-a836-918ec130dc0a migration_status: NULL replication_status: disabled replication_extended_status: NULL replication_driver_data: NULL consistencygroup_id: NULL provider_id: NULL multiattach: 0 previous_status: NULL cluster_name: NULL group_id: NULL service_uuid: 674de52f-1c9a-402f-88c9-6b79c91a4249 shared_targets: 1 1 row in set (0.00 sec) ``` So encryption_key_id is the value that we were looking for. Then we can simply get our secret store: ``` [TEST]root@zabbix-1:~# openstack secret get http://controller.tc.tester-pl.pl:9311/v1/secrets/b13d2017-e3e5-4f5f-a836-91... +---------------+----------------------------------------------------------------------------------------+ | Field | Value | +---------------+----------------------------------------------------------------------------------------+ | Secret href | http://controller.tc.tester-pl.pl:9311/v1/secrets/b13d2017-e3e5-4f5f-a836-91... | | Name | None | | Created | 2021-02-12T13:41:39+00:00 | | Status | ACTIVE | | Content types | {u'default': u'application/octet-stream'} | | Algorithm | aes | | Bit length | 512 | | Secret type | symmetric | | Mode | None | | Expiration | None | +---------------+----------------------------------------------------------------------------------------+ ``` And of course encryption key, by command(we will save it to file **my_symmetric_key.key**): ``` barbican secret get --payload_content_type application/octet-stream http://controller.tester-pl.pl:9311/v1/secrets/b13d2017-e3e5-4f5f-a836-918ec... --file my_symmetric_key.key ``` We need to transfer symmetric key to passphrase then: ``` [TEST]root@barbican-01:/var/log/barbican# hexdump -e '16/1 "%02x"' my_symmetric_key.key ``` The output is our LUKS Passphrase. We can go to our compute node and decrypt a volume: ``` [TEST]root@comp-09:/home/jwasilewski# cryptsetup luksOpen /dev/disk/by-id/wwn-0x6e00084100ee7e7e7fe79b5900003a89 my-encrypted-volume-decrypted Enter passphrase for /dev/disk/by-id/wwn-0x6e00084100ee7e7e7fe79b5900003a89: ``` Then we can confirm, that our volume is decrypted: ``` [TEST]root@comp-09:/home/jwasilewski# qemu-img info /dev/mapper/my-encrypted-volume-decrypted image: /dev/mapper/my-encrypted-volume-decrypted file format: raw virtual size: 20G (21472739328 bytes) disk size: 0 ``` Thanks again for sharing it, I believe it is a super-useful procedure. Best regards, Jan czw., 11 lut 2021 o 13:23 Lee Yarwood <lyarwood@redhat.com> napisał(a):
On 10-02-21 17:43:03, Lee Yarwood wrote:
On 10-02-21 11:29:06, Jan Wasilewski wrote:
Thank you for a nice description of how everything is organized. It is much easier to understand the full workflow.
I'll try to find some time to write these up again later in the week. That would be great, I will try to do this by myself, but I'm wondering if it's possible to do "all magic" directly from a payload that is visible from barbican CLI.
My thanks to gcharot for writing the following up downstream a while ago and highlighting some easy ways of achieving this.
The following assumes that the volume is already mapped and connected to the localhost, in this case I'm just using the LVs used by the default LVM/iSCSI c-vol backend in my devstack env.
It also assumes you have access to the secrets associated with the encrypted volume, by default admins do not.
- Starting with an encrypted volume
$ sudo qemu-img info --output=json /dev/stack-volumes-lvmdriver-1/volume-d4cc53db-6add-4c29-9f96-42a5498f8bd0 | jq .format "luks"
- Fetch and store the key locally
$ openstack secret get --payload_content_type 'application/octet-stream' http://192.168.122.208/key-manager/v1/secrets/6fd4f879-005d-4b7d-9e5f-2505f0... --file mysecret.key
- Use dmcrypt to decrypt the device using the key as a passphrase
$ yes $(hexdump -e '16/1 "%02x"' mysecret.key) | sudo cryptsetup luksOpen /dev/stack-volumes-lvmdriver-1/volume-d4cc53db-6add-4c29-9f96-42a5498f8bd0 volume-d4cc53db-6add-4c29-9f96-42a5498f8bd0
- This should leave you with the decrypted volume under /dev/mapper
$ sudo qemu-img info /dev/mapper/volume-d4cc53db-6add-4c29-9f96-42a5498f8bd0 image: /dev/mapper/volume-d4cc53db-6add-4c29-9f96-42a5498f8bd0 file format: raw virtual size: 0.998 GiB (1071644672 bytes) disk size: 0 B
Hope this helps!
-- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76