Failed live migration and duplicate attachments
Matt Riedemann
mriedemos at gmail.com
Thu Dec 20 22:33:36 UTC 2018
On 12/20/2018 8:35 AM, Torin Woltjer wrote:
> How would you recommend addressing the duplicate attachments? There
> doesn't seem to be any adverse effect of them being there, but I cannot
> detach them because they are root volumes.
These are records in the cinder "volume_attachment" table correct? If
so, there should be volume attachment records for both the source host
and the destination host for the live migration. There should be a
"connector" json blob in them and the ones to remove would be the ones
that have the destination host information (should at least have the
host IP). Since the live migration failed, you should only care about
the volume attachments for the source host.
Before deleting anything, cross-check that the
block_device_mappings.attachment_id in the nova (cell) database matches
the source host volume attachments in the cinder DB. nova-compute should
have made sure those BDM records were rolled back properly here:
https://github.com/openstack/nova/blob/a1c01f97ae0397e8f4ccff5eb2a8b8f5ef7f7221/nova/compute/manager.py#L6843
but you should double check. Actually, looking at that code, they are
likely still pointing at the destination volume attachments since we try
to delete the attachments before updating the block_device_mappings
table entries (we should probably reverse that order in the code).
When it comes to actually deleting those destination host volume
attachment records, I would use the cinder API:
https://developer.openstack.org/api-ref/block-storage/v3/#delete-attachment
Just to make sure something doesn't get screwed up if you try and do
manual database surgery (as noted, you might need to manually update the
block_device_mappings.attachment_id values in the nova DB to point at
the source host volume attachments for that instance).
--
Thanks,
Matt
More information about the openstack-discuss
mailing list