[nova][cinder][ops] question/confirmation of legacy vol attachment migration
Hello Cinderinos,
I've now got a working patch that migrates legacy volume attachments to new style v3 attachments [1]. The fun stuff showing it working is in this paste [2].
We want to do this data migration in nova because we still have a lot of compatibility code since Queens for pre-v3 style attachments and we can't remove that compatibility code (ever) if we don't first make sure we provide a data migration routine for operators to roll through. So for example if this lands in Ussuri we can can enforce a nova-status upgrade check in V and rip out code in X.
Without digging into the patch, this is the flow:
1. On nova-compute restart, query the nova DB for instances on the compute host with legacy volume attachments.
2. For each of those, create a new style attachment with the host connector and update the BlockDeviceMapping information in the nova DB (attachment_id and connection_info).
3. Delete the existing legacy attachment so when the server is deleted the volume status goes back to 'available' due to proper attachment reference counting in the Cinder DB.
My main question is on #3. Right now I'm calling the v3 attachment delete API rather than the v2 os-terminate_connection API. Is that sufficient to cleanup the legacy attachment on the storage backend even though the connection was created via os-initialize_connection originally? Looking at the cinder code, attachment_delete hits the connection terminate code under the covers [3]. So that looks OK. The only thing I can really think of is if a host connector is not provided or tracked with the legacy attachment, is that going to cause problems? Note that I think volume drivers are already required to deal with that today anyway because of the "local delete" scenario in the compute API where the compute host that the server is on is down and thus we don't have a host connector to provide to Cinder to terminate the connection.
So Cinder people, are you OK with this flow?
Hello Novaheads,
Do you have any issues with the above? Note the migration routine is threaded out on compute start so it doesn't block, similar to the ironic flavor data migration introduced in Pike.
One question I have is if we should add a config option for this so operators can enable/disable it as needed. Note that this requires nova to be configured with a service user that has the admin role to do this stuff in cinder since we don't have a user token, similar to nova doing things with neutron ports without a user token. Testing this with devstack requires [4]. By default [cinder]/auth_type is None and not required so by default this migration routine is not going to run so maybe that is sufficient?
Hello Operatorites,
Do you have any issues with what's proposed above?
[1] https://review.opendev.org/#/c/549130/ [2] http://paste.openstack.org/show/781063/ [3] https://github.com/openstack/cinder/blob/410791580ef60ddb03104bf20766859ed9d... [4] https://review.opendev.org/#/c/685488/
So looking into the cinder code, calling attachment_delete should be what we want to call. But.
I think if we don't have a host connector passed in and the attachment record doesn't have a connector saved, then that results in the volume manager not calling the cinder driver to terminate_connection and return. This also bypasses the driver's remove_export() which is the last chance for a driver to unexport a volume.
Walt
On Thu, Oct 3, 2019 at 7:27 PM Matt Riedemann mriedemos@gmail.com wrote:
Hello Cinderinos,
I've now got a working patch that migrates legacy volume attachments to new style v3 attachments [1]. The fun stuff showing it working is in this paste [2].
We want to do this data migration in nova because we still have a lot of compatibility code since Queens for pre-v3 style attachments and we can't remove that compatibility code (ever) if we don't first make sure we provide a data migration routine for operators to roll through. So for example if this lands in Ussuri we can can enforce a nova-status upgrade check in V and rip out code in X.
Without digging into the patch, this is the flow:
- On nova-compute restart, query the nova DB for instances on the
compute host with legacy volume attachments.
- For each of those, create a new style attachment with the host
connector and update the BlockDeviceMapping information in the nova DB (attachment_id and connection_info).
- Delete the existing legacy attachment so when the server is deleted
the volume status goes back to 'available' due to proper attachment reference counting in the Cinder DB.
My main question is on #3. Right now I'm calling the v3 attachment delete API rather than the v2 os-terminate_connection API. Is that sufficient to cleanup the legacy attachment on the storage backend even though the connection was created via os-initialize_connection originally? Looking at the cinder code, attachment_delete hits the connection terminate code under the covers [3]. So that looks OK. The only thing I can really think of is if a host connector is not provided or tracked with the legacy attachment, is that going to cause problems? Note that I think volume drivers are already required to deal with that today anyway because of the "local delete" scenario in the compute API where the compute host that the server is on is down and thus we don't have a host connector to provide to Cinder to terminate the connection.
So Cinder people, are you OK with this flow?
Hello Novaheads,
Do you have any issues with the above? Note the migration routine is threaded out on compute start so it doesn't block, similar to the ironic flavor data migration introduced in Pike.
One question I have is if we should add a config option for this so operators can enable/disable it as needed. Note that this requires nova to be configured with a service user that has the admin role to do this stuff in cinder since we don't have a user token, similar to nova doing things with neutron ports without a user token. Testing this with devstack requires [4]. By default [cinder]/auth_type is None and not required so by default this migration routine is not going to run so maybe that is sufficient?
Hello Operatorites,
Do you have any issues with what's proposed above?
[1] https://review.opendev.org/#/c/549130/ [2] http://paste.openstack.org/show/781063/ [3]
https://github.com/openstack/cinder/blob/410791580ef60ddb03104bf20766859ed9d... [4] https://review.opendev.org/#/c/685488/
--
Thanks,
Matt
On 10/4/2019 11:03 AM, Walter Boring wrote:
I think if we don't have a host connector passed in and the attachment record doesn't have a connector saved, then that results in the volume manager not calling the cinder driver to terminate_connection and return. This also bypasses the driver's remove_export() which is the last chance for a driver to unexport a volume.
Two things:
1. Yeah if the existing legacy attachment record doesn't have a connector I was worried about not properly cleaning on for that old connection, which is something I mentioned before, but also as mentioned we potentially have that case when a server is deleted and we can't get to the compute host to get the host connector, right?
2. If I were to use os-terminate_connection, I seem to have a tricky situation on the migration flow because right now I'm doing:
a) create new attachment with host connector b) complete new attachment (put the volume back to in-use status) - if this fails I attempt to delete the new attachment c) delete the legacy attachment - I intentionally left this until the end to make sure (a) and (b) were successful.
If I change (c) to be os-terminate_connection, will that screw up the accounting on the attachment created in (a)?
If I did the terminate_connection first (before creating a new attachment), could that leave a window of time where the volume is shown as not attached/in-use? Maybe not since it's not the begin_detaching/os-detach API...I'm fuzzy on the cinder volume state machine here.
Or maybe the flow would become:
a) create new attachment with host connector b) terminate the connection for the legacy attachment - if this fails, delete the new attachment created in (a) c) complete the new attachment created in (a) - if this fails...?
Without digging into the flow of a cold or live migration I want to say that's closer to what we do there, e.g. initialize_connection for the new host, terminate_connection for the old host, complete the new attachment.
On 04/10, Matt Riedemann wrote:
On 10/4/2019 11:03 AM, Walter Boring wrote:
I think if we don't have a host connector passed in and the attachment record doesn't have a connector saved, then that results in the volume manager not calling the cinder driver to terminate_connection and return. This also bypasses the driver's remove_export() which is the last chance for a driver to unexport a volume.
Two things:
- Yeah if the existing legacy attachment record doesn't have a connector I
was worried about not properly cleaning on for that old connection, which is something I mentioned before, but also as mentioned we potentially have that case when a server is deleted and we can't get to the compute host to get the host connector, right?
Hi,
Not really... In that case we still have the BDM info in the DB, so we can just make the 3 Cinder REST API calls ourselves (begin_detaching, terminate_connection and detach) to have the volume unmapped, the export removed, and the volume return to available as usual, without needing to go to the storage array manually.
- If I were to use os-terminate_connection, I seem to have a tricky
situation on the migration flow because right now I'm doing:
a) create new attachment with host connector b) complete new attachment (put the volume back to in-use status)
- if this fails I attempt to delete the new attachment
c) delete the legacy attachment - I intentionally left this until the end to make sure (a) and (b) were successful.
If I change (c) to be os-terminate_connection, will that screw up the accounting on the attachment created in (a)?
If I did the terminate_connection first (before creating a new attachment), could that leave a window of time where the volume is shown as not attached/in-use? Maybe not since it's not the begin_detaching/os-detach API...I'm fuzzy on the cinder volume state machine here.
Or maybe the flow would become:
a) create new attachment with host connector
This is a good idea in itself, but it's not taking into account weird behaviors that some Cinder drivers may have when you call them twice to initialize the connection on the same host. Some drivers end up creating a different mapping for the volume instead of returning the existing one; we've had bugs like this before, and that's why Nova made a change in its live instance migration code to not call intialize_connection on the source host to get the connection_info for detaching.
b) terminate the connection for the legacy attachment
- if this fails, delete the new attachment created in (a)
c) complete the new attachment created in (a)
- if this fails...?
Without digging into the flow of a cold or live migration I want to say that's closer to what we do there, e.g. initialize_connection for the new host, terminate_connection for the old host, complete the new attachment.
I think any workaround we try to find has a good chance of resulting in a good number of bugs.
In my opinion our options are:
1- Completely detach and re-attach the volume 2- Write new code in Cinder
The new code can be either a new action or we can just add a microversion to attachment create to also accept "connection_info", and when we provide connection_info on the call the method confirms that it's a "migration" (the volume is 'in-use' and doesn't have any attachments) and it doesn't bother to call the cinder-volume to export and map the volume again and simply saves this information in the DB.
I know the solution it's not "clean/nice/elegant", and I'd rather go with option 1, but that would be terrible user experience, so I'll settle for a solution that doesn't add much code to Cinder, is simple for Nova, and is less likely to result in bugs.
What do you think?
Regards, Gorka.
PS: In this week's meeting we briefly discussed this topic and agreed to continue the conversation here and retake it on next week's meeting.
--
Thanks,
Matt
On 10/10/2019 5:00 AM, Gorka Eguileor wrote:
- Yeah if the existing legacy attachment record doesn't have a connector I
was worried about not properly cleaning on for that old connection, which is something I mentioned before, but also as mentioned we potentially have that case when a server is deleted and we can't get to the compute host to get the host connector, right?
Hi,
Not really... In that case we still have the BDM info in the DB, so we can just make the 3 Cinder REST API calls ourselves (begin_detaching, terminate_connection and detach) to have the volume unmapped, the export removed, and the volume return to available as usual, without needing to go to the storage array manually.
I'm not sure what you mean. Yes we have the BDM in nova but if it's really old it won't have the host connector stashed away in the connection_info dict and we won't be able to pass that to the terminate_connection API:
https://github.com/openstack/nova/blob/19.0.0/nova/compute/api.py#L2186
Are you talking about something else? I realize ^ is very edge case since we've been storing the connector in the BDM.connection_info since I think at least Liberty or Mitaka.
- If I were to use os-terminate_connection, I seem to have a tricky
situation on the migration flow because right now I'm doing:
a) create new attachment with host connector b) complete new attachment (put the volume back to in-use status)
- if this fails I attempt to delete the new attachment
c) delete the legacy attachment - I intentionally left this until the end to make sure (a) and (b) were successful.
If I change (c) to be os-terminate_connection, will that screw up the accounting on the attachment created in (a)?
If I did the terminate_connection first (before creating a new attachment), could that leave a window of time where the volume is shown as not attached/in-use? Maybe not since it's not the begin_detaching/os-detach API...I'm fuzzy on the cinder volume state machine here.
Or maybe the flow would become:
a) create new attachment with host connector
This is a good idea in itself, but it's not taking into account weird behaviors that some Cinder drivers may have when you call them twice to initialize the connection on the same host. Some drivers end up creating a different mapping for the volume instead of returning the existing one; we've had bugs like this before, and that's why Nova made a change in its live instance migration code to not call intialize_connection on the source host to get the connection_info for detaching.
Huh...I thought attachments in cinder were a dime a dozen and you could create/delete them as needed, or that was the idea behind the new v3 attachments stuff. It seems to at least be what I remember John Griffith always saying we should be able to do.
Also if you can't refresh the connection info on the same host then a change like this:
https://review.opendev.org/#/c/579004/
Which does just that - refreshes the connection info doing reboot and start instance operations - would break on those volume drivers if I'm following you.
b) terminate the connection for the legacy attachment
- if this fails, delete the new attachment created in (a)
c) complete the new attachment created in (a)
- if this fails...?
Without digging into the flow of a cold or live migration I want to say that's closer to what we do there, e.g. initialize_connection for the new host, terminate_connection for the old host, complete the new attachment.
I think any workaround we try to find has a good chance of resulting in a good number of bugs.
In my opinion our options are:
1- Completely detach and re-attach the volume
I'd really like to avoid this if possible because it could screw up running applications and the migration operation itself is threaded out to not hold up the restart of the compute service. But maybe that's also true of what I've got written up today though it's closer to what we do during resize/cold migrate (though those of course involve downtime for the guest).
2- Write new code in Cinder
The new code can be either a new action or we can just add a microversion to attachment create to also accept "connection_info", and when we provide connection_info on the call the method confirms that it's a "migration" (the volume is 'in-use' and doesn't have any attachments) and it doesn't bother to call the cinder-volume to export and map the volume again and simply saves this information in the DB.
If the volume is in-use it would have attachments, so I'm not following you there. Even if the volume is attached the "legacy" way from a nova perspective, using os-initialize_connection, there is a volume attachment record in the cinder DB (I confirmed this in my devstack testing and the notes are in my patch). It's also precisely the problem I'm trying to solve which is without deleting the old legacy attachment, when you delete the server the volume is detached but still shows up in cinder as in-use because of the orphaned attachment.
I know the solution it's not "clean/nice/elegant", and I'd rather go with option 1, but that would be terrible user experience, so I'll settle for a solution that doesn't add much code to Cinder, is simple for Nova, and is less likely to result in bugs.
What do you think?
Regards, Gorka.
PS: In this week's meeting we briefly discussed this topic and agreed to continue the conversation here and retake it on next week's meeting.
Thanks for discussing it and getting back to me.
a) create new attachment with host connector
This is a good idea in itself, but it's not taking into account weird behaviors that some Cinder drivers may have when you call them twice to initialize the connection on the same host. Some drivers end up creating a different mapping for the volume instead of returning the existing one; we've had bugs like this before, and that's why Nova made a change in its live instance migration code to not call intialize_connection on the source host to get the connection_info for detaching.
Huh...I thought attachments in cinder were a dime a dozen and you could create/delete them as needed, or that was the idea behind the new v3 attachments stuff. It seems to at least be what I remember John Griffith always saying we should be able to do.
Also if you can't refresh the connection info on the same host then a change like this:
https://review.opendev.org/#/c/579004/
Which does just that - refreshes the connection info doing reboot and start instance operations - would break on those volume drivers if I'm following you.
Creating attachements, using the new attachments API, is a pretty low overhead thing. The issue is/was with the way Nova was calling initialize_connection expecting it to be an idempotent operation.
I think we've identified most drivers that had an issue with this. It wasn't a documented assumption on the Cinder side, so I remember when we first realized that was what Nova was doing, we found a lot of Cinder backends that had issues with this.
With initialize_connection, at least how it was intended, it is telling the backend to create a new connection between the storage and the host. So every time initialize_connection was called, most backends would make configuration changes on the storage backend to expose the volume to the requested host. Depending on how that backend worked, this could result in multiple separate (and different) connection settings for how the host can access the volume.
Most drivers are now aware of this (mis?)use of the call and will first check if there is an existing configuration and just return those settings if it's found. There's no real way to test and enforce that easily, so making sure all drivers including newly added ones behave that way has been up to cores remembering to look for it during code reviews.
But you can create as many attachment objects in the database as you want.
Sean
On 10/10/2019 2:20 PM, Sean McGinnis wrote:
Most drivers are now aware of this (mis?)use of the call and will first check if there is an existing configuration and just return those settings if it's found. There's no real way to test and enforce that easily, so making sure all drivers including newly added ones behave that way has been up to cores remembering to look for it during code reviews.
It's unrelated to what I'm trying to solve, but could a cinder tempest plugin test be added which hits the initialize_connection API multiple times without changing host connector and assert the driver is doing the correct thing, whatever that is? Maybe it's just asserting that the connection_info returned from the first call is identical to subsequent calls if the host connector dict input doesn't change?
But you can create as many attachment objects in the database as you want.
But you have to remember to delete them otherwise the volume doesn't leave in-use status even if the volume is detached from the server, so there is attachment counting that needs to happen somewhere (I know cinder does it, but part of that is also on the client side - nova in this case). With the legacy attach flow nova would being_detaching, terminate_connection and then call os-detach and I suppose os-detach could puke if the client hadn't done the attachment cleanup properly, i.e. hadn't called terminate_connection. With the v3 attachments flow we don't have that, we just create attachment, update it with host connector and then complete it. On detach we just delete the attachment and if it's the last one the volume is no longer in-use. I'm not advocating adding another os-detach-like API for the v3 attachment flow, just saying it's an issue if the client isn't aware of all that.
On 10/10, Matt Riedemann wrote:
On 10/10/2019 5:00 AM, Gorka Eguileor wrote:
- Yeah if the existing legacy attachment record doesn't have a connector I
was worried about not properly cleaning on for that old connection, which is something I mentioned before, but also as mentioned we potentially have that case when a server is deleted and we can't get to the compute host to get the host connector, right?
Hi,
Not really... In that case we still have the BDM info in the DB, so we can just make the 3 Cinder REST API calls ourselves (begin_detaching, terminate_connection and detach) to have the volume unmapped, the export removed, and the volume return to available as usual, without needing to go to the storage array manually.
I'm not sure what you mean. Yes we have the BDM in nova but if it's really old it won't have the host connector stashed away in the connection_info dict and we won't be able to pass that to the terminate_connection API:
https://github.com/openstack/nova/blob/19.0.0/nova/compute/api.py#L2186
Are you talking about something else? I realize ^ is very edge case since we've been storing the connector in the BDM.connection_info since I think at least Liberty or Mitaka.
Hi,
I didn't know that Nova didn't use to store the connector... For those cases it is definitely going to be a problem.
If you have one such cases, and the Nova compute node is down (so you cannot get the connector info), then we should just wait until the node is back up to do the migration.
- If I were to use os-terminate_connection, I seem to have a tricky
situation on the migration flow because right now I'm doing:
a) create new attachment with host connector b) complete new attachment (put the volume back to in-use status)
- if this fails I attempt to delete the new attachment
c) delete the legacy attachment - I intentionally left this until the end to make sure (a) and (b) were successful.
If I change (c) to be os-terminate_connection, will that screw up the accounting on the attachment created in (a)?
If I did the terminate_connection first (before creating a new attachment), could that leave a window of time where the volume is shown as not attached/in-use? Maybe not since it's not the begin_detaching/os-detach API...I'm fuzzy on the cinder volume state machine here.
Or maybe the flow would become:
a) create new attachment with host connector
This is a good idea in itself, but it's not taking into account weird behaviors that some Cinder drivers may have when you call them twice to initialize the connection on the same host. Some drivers end up creating a different mapping for the volume instead of returning the existing one; we've had bugs like this before, and that's why Nova made a change in its live instance migration code to not call intialize_connection on the source host to get the connection_info for detaching.
Huh...I thought attachments in cinder were a dime a dozen and you could create/delete them as needed, or that was the idea behind the new v3 attachments stuff. It seems to at least be what I remember John Griffith always saying we should be able to do.
Sure, you can create them freely, but the old and new API's were not meant to be mixed, which is what we would be doing here.
The more I look at this, the more I think it is a bad idea.
First, when you create the new attachment on a volume that was attached using the old APIs (Cinder attachment exists in DB without connection info):
- Cinder DB attachment entry has the instance_uuid: then, since the volume to be migrated is not multi-attach, we will reuse the attachment that already exists [1] and a new one will not be created. So we will end up with just 1 attachment entry like if we didn't do anything.
- If the DB attachment entry doesn't have the instance_uuid, then the attachment creation will fail [2] because it is not a multi-attach volume.
If somehow we were able to create a second attachment entry, then the attachment_update will raise an exception [3], because it's expecting to have the connector information in all the attachments (because old and new attachment flows were not meant to be mixed).
Even if we pass through that, this is not a multi-attach volume, so it will still fail because it cannot have 2 attachments [4].
Even if we get past that, when we create the attachment with the connector info or create it and then update it with the connection info, we'll get a new call to initialize_connection, and depending on the driver this will create a new export and mapping or reuse the old one.
- If we create a new one the code could be fine when you call Cinder to remove that attachment, because we have 2 exports and we'll remove one. "Hopefully" the one we want.
- The problem is if the driver's initialize_connection was idempotent, because then the new attach API expects it to return "True" when called a second time to initialize_connection with the same connector info, yet I don't think this was documented anywhere [5], so I don't think there are any drivers that are doing this. If drivers are idempotent and they don't return "True", then when we terminate the old attach connection we'll remove the export that is used by both connections, breaking the attachment.
And this is not even thinking of what will happen on the OS-Brick side, which is most likely not good. For example, if we have a multipath iSCSI volume and the driver created a new target-portal, then the new devices that will appear on the system will be aggregated to the already existing multipath DM, which means that the terminate connection of the first one will flush the shared multipath DM, thus destroying the mapping from the new API flow.
I stand by my initial recommendation, being able to update the existing attachment to add the connection information from Nova.
Cheers, Gorka.
[1]: https://opendev.org/openstack/cinder/src/commit/b8198de09aa13113d16d0cef8916... [2]: https://opendev.org/openstack/cinder/src/commit/b8198de09aa13113d16d0cef8916... [3]: https://opendev.org/openstack/cinder/src/commit/b8198de09aa13113d16d0cef8916... [4]: https://opendev.org/openstack/cinder/src/commit/b8198de09aa13113d16d0cef8916... [5]: https://opendev.org/openstack/cinder/src/commit/b8198de09aa13113d16d0cef8916...
Also if you can't refresh the connection info on the same host then a change like this:
https://review.opendev.org/#/c/579004/
Which does just that - refreshes the connection info doing reboot and start instance operations - would break on those volume drivers if I'm following you.
The part related to the new API looks fine, but doing that for the old initialize_connection doesn't look right to me.
Cheers, Gorka.
b) terminate the connection for the legacy attachment
- if this fails, delete the new attachment created in (a)
c) complete the new attachment created in (a)
- if this fails...?
Without digging into the flow of a cold or live migration I want to say that's closer to what we do there, e.g. initialize_connection for the new host, terminate_connection for the old host, complete the new attachment.
I think any workaround we try to find has a good chance of resulting in a good number of bugs.
In my opinion our options are:
1- Completely detach and re-attach the volume
I'd really like to avoid this if possible because it could screw up running applications and the migration operation itself is threaded out to not hold up the restart of the compute service. But maybe that's also true of what I've got written up today though it's closer to what we do during resize/cold migrate (though those of course involve downtime for the guest).
I agree, this is not an idea we should pursue.
2- Write new code in Cinder
The new code can be either a new action or we can just add a microversion to attachment create to also accept "connection_info", and when we provide connection_info on the call the method confirms that it's a "migration" (the volume is 'in-use' and doesn't have any attachments) and it doesn't bother to call the cinder-volume to export and map the volume again and simply saves this information in the DB.
If the volume is in-use it would have attachments, so I'm not following you there. Even if the volume is attached the "legacy" way from a nova perspective, using os-initialize_connection, there is a volume attachment record in the cinder DB (I confirmed this in my devstack testing and the notes are in my patch). It's also precisely the problem I'm trying to solve which is without deleting the old legacy attachment, when you delete the server the volume is detached but still shows up in cinder as in-use because of the orphaned attachment.
I know the solution it's not "clean/nice/elegant", and I'd rather go with option 1, but that would be terrible user experience, so I'll settle for a solution that doesn't add much code to Cinder, is simple for Nova, and is less likely to result in bugs.
What do you think?
Regards, Gorka.
PS: In this week's meeting we briefly discussed this topic and agreed to continue the conversation here and retake it on next week's meeting.
Thanks for discussing it and getting back to me.
--
Thanks,
Matt
On 10/17/2019 5:24 AM, Gorka Eguileor wrote:
I stand by my initial recommendation, being able to update the existing attachment to add the connection information from Nova.
OK, thanks for the input and thoughtfulness on this. I've abandoned my change since I'm not going to be pushing this boulder anymore but left notes in the change in case someone else wants to pick it up some day.
Note to nova cores: this means we'll have legacy volume attachment compat code around forever.
On Fri, Nov 22, 2019 at 3:54 AM Matt Riedemann mriedemos@gmail.com wrote:
On 10/17/2019 5:24 AM, Gorka Eguileor wrote:
I stand by my initial recommendation, being able to update the existing attachment to add the connection information from Nova.
OK, thanks for the input and thoughtfulness on this. I've abandoned my change since I'm not going to be pushing this boulder anymore but left notes in the change in case someone else wants to pick it up some day.
Note to nova cores: this means we'll have legacy volume attachment compat code around forever.
--
Thanks,
Matt
Hi Groka, Cinder & Nova devs,
Following up this thread from the context of https://review.opendev.org/#/c/579004/
To summarise the discussion: - initialize_connection shouldn't be called more than once, as it may mess up some drivers. - To (safely) refresh connection_info a new api for cinder is required. - a patch to nova, such as #579004 could make a call to this new api to refresh connection info on events such as reboot.
Is there any other context to this issue I've missed or an alternate path to solving this?
Does the creation of a new api for cinder have any implications for being able to backport this patch for nova?
What would be the process for me to kick off this work?
Thanks for your help, Brett (bcm)
On 12/5/2019 10:19 PM, Brett Milford wrote:
Does the creation of a new api for cinder have any implications for being able to backport this patch for nova?
The nova code would need to be tolerant of the cinder API microversion not being available. That could probably use the nova.volume.cinder.is_microversion_supported method. The Cinder API change would not be backportable though.
What would be the process for me to kick off this work?
I'm assuming you'd start with a cinder spec since it's a versioned API change over there.
participants (5)
-
Brett Milford
-
Gorka Eguileor
-
Matt Riedemann
-
Sean McGinnis
-
Walter Boring