To whom it may concern
Hi. I encountered an issue with OpenStack where volume information was duplicated when Live Migration failed. Could you please provide a solution to this issue?
[Issue]
After a Live Migration failure, when I checked the volume where the error occurred using the 'openstack volume list' command, the following was displayed:
+--------------------------------------+-----------------------------------+-----------+------+---------------------------------------------------------------------------------------------------------------------------------------------+ | ID | Name | Status | Size | Attached to | +--------------------------------------+-----------------------------------+-----------+------+---------------------------------------------------------------------------------------------------------------------------------------------+ | 9ba9734f-575e-41de-8bb4-60839388e0ad | | in-use | 10 | Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda |
I expect that the 'Attached to' display should only appear once as there is one volume connected, but it is duplicated for each failed attempt. The versions of OpenStack and nova are as follows:
OpenStack Zed
$ openstack --version openstack 6.0.0 $ nova --version nova CLI is deprecated and will be a removed in a future release 18.1.0
[Details]
The volume display is duplicated under the following conditions:
- When Live Migration of an instance fails
If the communication between the ports required for Live Migration is closed, Live Migration will fail when executed.
$ sudo netstat -tuln | grep 16509 $
The output of nova-compute.log at the time of Live Migration failure is as follows:
--- 2024-02-28 17:04:32.119 2785 ERROR nova.virt.libvirt.driver [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] Live Migration failure: unable to connect to server at 'XXXXX.com:16509': Connection refused: libvirt.libvirtError: unable to connect to server at 'XXXXX.com:16509': Connection refused 2024-02-28 17:04:32.119 2785 DEBUG nova.virt.libvirt.driver [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] Migration operation thread notification thread_finished /usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py:10516 2024-02-28 17:04:32.607 2785 DEBUG nova.virt.libvirt.migration [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] VM running on src, migration failed _log /usr/lib/python3.9/site-packages/nova/virt/libvirt/migration.py:432 2024-02-28 17:04:32.608 2785 DEBUG nova.virt.libvirt.driver [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] Fixed incorrect job type to be 4 _live_migration_monitor /usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py:10330 2024-02-28 17:04:32.608 2785 ERROR nova.virt.libvirt.driver [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] Migration operation has aborted ---
After Live Migration, when I check the volume where the error occurred using the 'openstack volume list' command, the following is displayed:
+--------------------------------------+-----------------------------------+-----------+------+---------------------------------------------------------------------------------------------------------------------------------------------+ | ID | Name | Status | Size | Attached to | +--------------------------------------+-----------------------------------+-----------+------+---------------------------------------------------------------------------------------------------------------------------------------------+ | 9ba9734f-575e-41de-8bb4-60839388e0ad | | in-use | 10 | Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda |
'73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda' is displayed as many times as Live Migration failed. The duplicate display did not disappear even after Live Migration was successful.
Thanks,