Hi Rajat, First of all, thank you for your detailed response. Thanks to that, I was able to identify the source of the issue[1], which appears to be that this functionality is not yet implemented. I'm wondering what kind of workaround might exist. For volumes like vdb, vdc, etc., we can simply detach the volume from the virtual machine, but it’s not as straightforward for a root volume (vda). Is there a way to allow Cinder to handle such decryption without involving Nova in the process? Is there a simple way to "unplug" the root volume without destroying the VM? I’d like to keep the ports and other settings intact. I can turn off the VM if necessary, but I’d prefer to avoid making more drastic changes. Thanks again for your help and guidance. /JW [1] https://paste.openstack.org/show/b8lDJp2yPSC1nnAk2M0U/ sob., 14 wrz 2024 o 08:58 Rajat Dhasmana <rdhasman@redhat.com> napisał(a):

Hi Jan,

On Tue, Sep 10, 2024 at 5:53 PM Jan Wasilewski <finarffin@gmail.com> wrote:

...

Hi,

I would like to ask about the procedure to retype a Cinder volume that is in-use (bootable volume, connected to a VM) from an encrypted (LUKS) type to a non-encrypted type (1000 IOPS). I tried to do this via the CLI[1], and while it seemed to work (at least according to the CLI output), the process didn’t complete successfully. Cinder logs[2] indicate that everything looks more or less fine: the volume is created, the volume migration is OK, and the retype is marked as completed.

Retype of attached/in-use volumes happen on the nova side and cinder just sends a request with old and new volume to nova to copy data and swap them. So cinder logs might not be very relevant for this case, better would be to check nova logs with DEBUG enabled.

...

However, in the end, it doesn’t work (final output of [1]).

I reviewed the blueprint at https://blueprints.launchpad.net/cinder/+spec/retype-encrypted-volume, which appears to be implemented. Am I doing something wrong, or does this functionality not work as expected?

The blueprint[1] has the following spec[2] which has a link to nova side changes which never merged so I'm suspecting the work never got completed? Or maybe it's addressed in some other patch but I couldn't find details in the LP bug, the review comments on the patch and neither on the BP or spec. It would be good to check the errors on nova-compute logs to see where it is specifically failing.

[1] https://blueprints.launchpad.net/cinder/+spec/retype-encrypted-volume [2] https://review.opendev.org/c/openstack/cinder-specs/+/248593/20/specs/newton... [3] https://review.opendev.org/c/openstack/nova/+/252809

Thanks Rajat Dhasmana

Thanks in advance for your help and guidance.

...

/JW

*[1] Retype from CLI: https://paste.openstack.org/show/bVpFT82oURnanXkixreT/ <https://paste.openstack.org/show/bVpFT82oURnanXkixreT/>[2] cinder-logs: https://paste.openstack.org/show/b5rZpWNMqdi6qdpjaDfg/ <https://paste.openstack.org/show/b5rZpWNMqdi6qdpjaDfg/>*

Hi, First of all, thank you for your response. I tried to follow your instructions, but it didn’t work, as we cannot retype a volume that is in a status other than 'in-use' or 'available' [1]. The 'reserved' status is not valid for this operation. To bypass this, I tried to be a bit clever and manually changed the volume status (as an admin) to 'available'[2]. Unfortunately, I encountered an error during the retype process [3]. It seems that the Nova part is handled, but now I’m blocked on the Cinder side, and we cannot simply shelve the VM. I’m wondering if there’s still a workaround for this or if this approach is just not feasible. /JW [1] https://docs.openstack.org/cinder/latest/contributor/migration.html [2] https://paste.openstack.org/show/bkxt8dLdalfCNEeduV6j/ [3] https://paste.openstack.org/show/bkGFXrLjlQvMlkbrJUZ1/ pt., 20 wrz 2024 o 13:04 <smooney@redhat.com> napisał(a):

...

Hi Rajat,

First of all, thank you for your detailed response. Thanks to that, I was able to identify the source of the issue[1], which appears to be that

On Fri, 2024-09-20 at 12:47 +0200, Jan Wasilewski wrote: this

...

functionality is not yet implemented.

I'm wondering what kind of workaround might exist. For volumes like vdb, vdc, etc., we can simply detach the volume from the virtual machine, but it’s not as straightforward for a root volume (vda). Is there a way to allow Cinder to handle such decryption without involving Nova in the process? Is there a simple way to "unplug" the root volume without destroying the VM? no we do not supprot detaching the root volume

you might and i stress might be able to shleve the instance and try and do the retrye then

when we shleve the volume shoudl change form inuse to reserved and it will obviouly not be attached to a nova host once the instnace reaches the shelve_offloaded state.

...

I’d like to keep the ports and other settings intact. I can turn off the VM if necessary, but I’d prefer to avoid making more drastic changes. shelve and unshelve will maintian all ports,volumes ectra so i think that should work for you.,

Thanks again for your help and guidance.

/JW

[1] https://paste.openstack.org/show/b8lDJp2yPSC1nnAk2M0U/

sob., 14 wrz 2024 o 08:58 Rajat Dhasmana <rdhasman@redhat.com> napisał(a):

...

Hi Jan,

On Tue, Sep 10, 2024 at 5:53 PM Jan Wasilewski <finarffin@gmail.com> wrote:

...

Hi,

I would like to ask about the procedure to retype a Cinder volume

...

...

...

in-use (bootable volume, connected to a VM) from an encrypted (LUKS) type to a non-encrypted type (1000 IOPS). I tried to do this via the CLI[1], and while it seemed to work (at least according to the CLI output), the

...

...

...

didn’t complete successfully. Cinder logs[2] indicate that everything looks more or less fine: the volume is created, the volume migration is OK, and the retype is marked as completed.

Retype of attached/in-use volumes happen on the nova side and cinder just sends a request with old and new volume to nova to copy data and swap

...

...

So cinder logs might not be very relevant for this case, better would be to check nova logs with DEBUG enabled.

...

However, in the end, it doesn’t work (final output of [1]).

I reviewed the blueprint at

https://blueprints.launchpad.net/cinder/+spec/retype-encrypted-volume,

...

which appears to be implemented. Am I doing something wrong, or does

...

...

...

functionality not work as expected?

The blueprint[1] has the following spec[2] which has a link to nova side changes which never merged so I'm suspecting the work never got completed? Or maybe it's addressed in some other patch but I couldn't find

in that state i would expect the voluem retype to happen on the cinder backend side but i dont know if cinder supports that. that is process them. this details in

...

...

the LP bug, the review comments on the patch and neither on the BP or spec. It would be good to check the errors on nova-compute logs to see where it is specifically failing.

[1] https://blueprints.launchpad.net/cinder/+spec/retype-encrypted-volume [2]

https://review.opendev.org/c/openstack/cinder-specs/+/248593/20/specs/newton...

...

[3] https://review.opendev.org/c/openstack/nova/+/252809

Thanks Rajat Dhasmana

Thanks in advance for your help and guidance.

...

/JW

*[1] Retype from CLI: https://paste.openstack.org/show/bVpFT82oURnanXkixreT/ <https://paste.openstack.org/show/bVpFT82oURnanXkixreT/>[2] cinder-logs: https://paste.openstack.org/show/b5rZpWNMqdi6qdpjaDfg/ <https://paste.openstack.org/show/b5rZpWNMqdi6qdpjaDfg/>*

On Fri, 2024-09-20 at 06:09 -0700, Alan Bishop wrote:

On Fri, Sep 20, 2024 at 5:48 AM Jan Wasilewski <finarffin@gmail.com> wrote:

...

Hi,

First of all, thank you for your response. I tried to follow your instructions, but it didn’t work, as we cannot retype a volume that is in a status other than 'in-use' or 'available' [1]. The 'reserved' status is not valid for this operation. To bypass this, I tried to be a bit clever and manually changed the volume status (as an admin) to 'available'[2]. Unfortunately, I encountered an error during the retype process [3].

It seems that the Nova part is handled, but now I’m blocked on the Cinder side, and we cannot simply shelve the VM. I’m wondering if there’s still a workaround for this or if this approach is just not feasible.

The problem is that cinder maintains a volume attachment record for the shelved instance, and resetting the volume status to 'available' doesn't overshadow the attachment record. Your pastebin [2] L3 is seems to be coming from the cinder code at [4]. See what "openstack volume attachment list" reports.

Unfortunately I don't think there's a way to bypass the existence of an attachment record, and I don't think hacking the DB to temporarily delete the record and restore it later would be advisable.

[4] https://opendev.org/openstack/cinder/src/branch/stable/2023.1/cinder/volume/... actully we might already have a bug for that

we do have one for neutorn. on the neutron side we are ment to unbind the port bug keep it owned by hte instnace when we shelve we shoudl be removing the attachment when we shelve too altought the volume shoudl still be reserved by the instance. im not sure if we can do that with cidner current api has both a tracker f or what isntance it belongs to and what host its bound too. if we have that split for cider we could remove the attachment (as it not exported to any host) while keepign the ownership of the volume and assotaion to the vm. we need to create a new attachment when we unshelve anyway.

Alan

/JW

...

[1] https://docs.openstack.org/cinder/latest/contributor/migration.html [2] https://paste.openstack.org/show/bkxt8dLdalfCNEeduV6j/ [3] https://paste.openstack.org/show/bkGFXrLjlQvMlkbrJUZ1/

pt., 20 wrz 2024 o 13:04 <smooney@redhat.com> napisał(a):

...

...

Hi Rajat,

First of all, thank you for your detailed response. Thanks to that, I was able to identify the source of the issue[1], which appears to be that

On Fri, 2024-09-20 at 12:47 +0200, Jan Wasilewski wrote: this

...

functionality is not yet implemented.

I'm wondering what kind of workaround might exist. For volumes like vdb, vdc, etc., we can simply detach the volume from the virtual machine, but it’s not as straightforward for a root volume (vda). Is there a way to allow Cinder to handle such decryption without involving Nova in the process? Is there a simple way to "unplug" the root volume without destroying the VM? no we do not supprot detaching the root volume

you might and i stress might be able to shleve the instance and try and do the retrye then

when we shleve the volume shoudl change form inuse to reserved and it will obviouly not be attached to a nova host once the instnace reaches the shelve_offloaded state.

...

 I’d like to keep the ports and other settings intact. I can turn off the VM if necessary, but I’d prefer to avoid making more drastic changes. shelve and unshelve will maintian all ports,volumes ectra so i think that should work for you.,

Thanks again for your help and guidance.

/JW

[1] https://paste.openstack.org/show/b8lDJp2yPSC1nnAk2M0U/

sob., 14 wrz 2024 o 08:58 Rajat Dhasmana <rdhasman@redhat.com> napisał(a):

...

Hi Jan,

On Tue, Sep 10, 2024 at 5:53 PM Jan Wasilewski <finarffin@gmail.com> wrote:

...

Hi,

I would like to ask about the procedure to retype a Cinder volume

...

...

...

in-use (bootable volume, connected to a VM) from an encrypted (LUKS) type to a non-encrypted type (1000 IOPS). I tried to do this via the CLI[1], and while it seemed to work (at least according to the CLI output), the

...

...

...

didn’t complete successfully. Cinder logs[2] indicate that everything looks more or less fine: the volume is created, the volume migration is OK, and the retype is marked as completed.

Retype of attached/in-use volumes happen on the nova side and cinder just sends a request with old and new volume to nova to copy data and swap

...

...

So cinder logs might not be very relevant for this case, better would be to check nova logs with DEBUG enabled.

...

However, in the end, it doesn’t work (final output of [1]).

I reviewed the blueprint at

https://blueprints.launchpad.net/cinder/+spec/retype-encrypted-volume,

...

which appears to be implemented. Am I doing something wrong, or does this functionality not work as expected?

The blueprint[1] has the following spec[2] which has a link to nova side changes which never merged so I'm suspecting the work never got completed? Or maybe it's addressed in some other patch but I couldn't find

in that state i would expect the voluem retype to happen on the cinder backend side but i dont know if cinder supports that. that is process them. details in

...

...

the LP bug, the review comments on the patch and neither on the BP or spec. It would be good to check the errors on nova-compute logs to see where it is specifically failing.

[1] https://blueprints.launchpad.net/cinder/+spec/retype-encrypted-volume [2]

https://review.opendev.org/c/openstack/cinder-specs/+/248593/20/specs/newton...

...

[3] https://review.opendev.org/c/openstack/nova/+/252809

Thanks Rajat Dhasmana

Thanks in advance for your help and guidance.

...

/JW

*[1] Retype from CLI: https://paste.openstack.org/show/bVpFT82oURnanXkixreT/ <https://paste.openstack.org/show/bVpFT82oURnanXkixreT/>[2] cinder-logs: https://paste.openstack.org/show/b5rZpWNMqdi6qdpjaDfg/ <https://paste.openstack.org/show/b5rZpWNMqdi6qdpjaDfg/>*