Failed live migration and duplicate attachments

Matt Riedemann mriedemos at gmail.com
Thu Dec 20 22:33:36 UTC 2018


On 12/20/2018 8:35 AM, Torin Woltjer wrote:
> How would you recommend addressing the duplicate attachments? There 
> doesn't seem to be any adverse effect of them being there, but I cannot 
> detach them because they are root volumes.

These are records in the cinder "volume_attachment" table correct? If 
so, there should be volume attachment records for both the source host 
and the destination host for the live migration. There should be a 
"connector" json blob in them and the ones to remove would be the ones 
that have the destination host information (should at least have the 
host IP). Since the live migration failed, you should only care about 
the volume attachments for the source host.

Before deleting anything, cross-check that the 
block_device_mappings.attachment_id in the nova (cell) database matches 
the source host volume attachments in the cinder DB. nova-compute should 
have made sure those BDM records were rolled back properly here:

https://github.com/openstack/nova/blob/a1c01f97ae0397e8f4ccff5eb2a8b8f5ef7f7221/nova/compute/manager.py#L6843

but you should double check. Actually, looking at that code, they are 
likely still pointing at the destination volume attachments since we try 
to delete the attachments before updating the block_device_mappings 
table entries (we should probably reverse that order in the code).

When it comes to actually deleting those destination host volume 
attachment records, I would use the cinder API:

https://developer.openstack.org/api-ref/block-storage/v3/#delete-attachment

Just to make sure something doesn't get screwed up if you try and do 
manual database surgery (as noted, you might need to manually update the 
block_device_mappings.attachment_id values in the nova DB to point at 
the source host volume attachments for that instance).

-- 

Thanks,

Matt



More information about the openstack-discuss mailing list