<div dir="ltr">Additional logs:<div><br></div><div>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [-] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Pre live migration failed at <a href="http://overcloud-novacompute-4.bdxworld.com">overcloud-novacompute-4.bdxworld.com</a>: oslo_messaging.rpc.client.RemoteError: Remote error: VolumeDeviceNotFound Volume device not found at .<br>['Traceback (most recent call last):\n', ' File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', ' File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 72, in wrapped\n context, exc, binary)\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 63, in wrapped\n return f(self, context, *args, **kw)\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/utils.py", line 1437, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 212, in decorated_function\n kwargs[\'instance\'], e, sys.exc_info())\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 200, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8288, in pre_live_migration\n bdm.save()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8249, in pre_live_migration\n migrate_data)\n', ' File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 10343, in pre_live_migration\n self._connect_volume(context, connection_info, instance)\n', ' File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1808, in _connect_volume\n vol_driver.connect_volume(connection_info, instance)\n', ' File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/volume/iscsi.py", line 64, in connect_volume\n device_info = self.connector.connect_volume(connection_info[\'data\'])\n', ' File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 154, in trace_logging_wrapper\n return f(*args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 391, in inner\n return f(*args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 524, in connect_volume\n self._cleanup_connection(connection_properties, force=True)\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 518, in connect_volume\n return self._connect_single_volume(connection_properties)\n', ' File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 78, in _wrapper\n return r(f, *args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 409, in call\n do = self.iter(retry_state=retry_state)\n', ' File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 368, in iter\n raise retry_exc.reraise()\n', ' File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 186, in reraise\n raise self.last_attempt.result()\n', ' File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result\n return self.__get_result()\n', ' File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result\n raise self._exception\n', ' File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 412, in call\n result = fn(*args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 597, in _connect_single_volume\n raise exception.VolumeDeviceNotFound(device=\'\')\n', 'os_brick.exception.VolumeDeviceNotFound: Volume device not found at .\n'].<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Traceback (most recent call last):<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8421, in _do_pre_live_migration_from_source<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] block_migration, disk, dest, migrate_data)<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] File "/usr/lib/python3.6/site-packages/nova/compute/rpcapi.py", line 947, in pre_live_migration<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] **msg_args)<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] retry=self.retry, transport_options=self.transport_options)<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] transport_options=transport_options)<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 690, in send<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] transport_options=transport_options)<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 680, in _send<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] raise result<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] oslo_messaging.rpc.client.RemoteError: Remote error: VolumeDeviceNotFound Volume device not found at .<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] ['Traceback (most recent call last):\n', ' File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', ' File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 72, in wrapped\n context, exc, binary)\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 63, in wrapped\n return f(self, context, *args, **kw)\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/utils.py", line 1437, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 212, in decorated_function\n kwargs[\'instance\'], e, sys.exc_info())\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 200, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8288, in pre_live_migration\n bdm.save()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8249, in pre_live_migration\n migrate_data)\n', ' File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 10343, in pre_live_migration\n self._connect_volume(context, connection_info, instance)\n', ' File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1808, in _connect_volume\n vol_driver.connect_volume(connection_info, instance)\n', ' File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/volume/iscsi.py", line 64, in connect_volume\n device_info = self.connector.connect_volume(connection_info[\'data\'])\n', ' File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 154, in trace_logging_wrapper\n return f(*args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 391, in inner\n return f(*args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 524, in connect_volume\n self._cleanup_connection(connection_properties, force=True)\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 518, in connect_volume\n return self._connect_single_volume(connection_properties)\n', ' File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 78, in _wrapper\n return r(f, *args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 409, in call\n do = self.iter(retry_state=retry_state)\n', ' File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 368, in iter\n raise retry_exc.reraise()\n', ' File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 186, in reraise\n raise self.last_attempt.result()\n', ' File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result\n return self.__get_result()\n', ' File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result\n raise self._exception\n', ' File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 412, in call\n result = fn(*args, **kwargs)\n', ' File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 597, in _connect_single_volume\n raise exception.VolumeDeviceNotFound(device=\'\')\n', 'os_brick.exception.VolumeDeviceNotFound: Volume device not found at .\n'].<br>2023-08-21 09:22:34.353 7 ERROR nova.compute.manager [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452]<br></div><div><br></div><div>With regards,</div><div>Swogat Pradhan</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 21, 2023 at 3:14 PM Swogat Pradhan <<a href="mailto:swogatpradhan22@gmail.com">swogatpradhan22@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div>I have an instance that has disk /dev/vda present in ceph and the rest of the 6 volumes (10 TB) each on netapp.</div><div>I am trying to migrate the VM to another host but it is failing to migrate with error volume not found like below:</div><div><br></div><div>2023-08-21 09:22:31.852 7 WARNING os_brick.initiator.connectors.iscsi [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] Couldn't find iSCSI nodes because iscsiadm err: iscsiadm: No records found<br>: os_brick.exception.VolumeDeviceNotFound: Volume device not found at .<br>2023-08-21 09:22:31.866 7 WARNING os_brick.initiator.connectors.iscsi [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.<br>: os_brick.exception.VolumeDeviceNotFound: Volume device not found at .<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] Exception during message handling: os_brick.exception.VolumeDeviceNotFound: Volume device not found at .<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server Traceback (most recent call last):<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 72, in wrapped<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server context, exc, binary)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self.force_reraise()<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.value<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 63, in wrapped<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/utils.py", line 1437, in decorated_function<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 212, in decorated_function<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info())<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self.force_reraise()<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.value<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 200, in decorated_function<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8288, in pre_live_migration<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server bdm.save()<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self.force_reraise()<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.value<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8249, in pre_live_migration<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server migrate_data)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 10343, in pre_live_migration<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self._connect_volume(context, connection_info, instance)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1808, in _connect_volume<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server vol_driver.connect_volume(connection_info, instance)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/volume/iscsi.py", line 64, in connect_volume<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server device_info = self.connector.connect_volume(connection_info['data'])<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 154, in trace_logging_wrapper<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 391, in inner<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 524, in connect_volume<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self._cleanup_connection(connection_properties, force=True)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self.force_reraise()<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.value<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 518, in connect_volume<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return self._connect_single_volume(connection_properties)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 78, in _wrapper<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return r(f, *args, **kwargs)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 409, in call<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server do = self.iter(retry_state=retry_state)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 368, in iter<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise retry_exc.reraise()<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 186, in reraise<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.last_attempt.result()<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return self.__get_result()<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self._exception<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 412, in call<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server result = fn(*args, **kwargs)<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 597, in _connect_single_volume<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise exception.VolumeDeviceNotFound(device='')<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server os_brick.exception.VolumeDeviceNotFound: Volume device not found at .<br>2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server<br>2023-08-21 09:22:34.678 7 INFO nova.virt.block_device [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Attempting to driver detach volume a005a015-0bb2-498e-9068-ac14570a5f2e from mountpoint /dev/vda<br>2023-08-21 09:22:34.680 7 WARNING nova.virt.block_device [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Detaching volume from unknown instance<br>2023-08-21 09:22:34.682 7 WARNING nova.virt.libvirt.driver [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] During detach_volume, instance disappeared.: nova.exception.InstanceNotFound: Instance 5520dbd7-8f12-4041-bd8c-65b4b55b0452 could not be found.<br>2023-08-21 09:22:34.707 7 INFO nova.virt.block_device [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Attempting to driver detach volume 01b9b430-9cfd-43b1-b674-b6fb830ee511 from mountpoint /dev/vdb<br>2023-08-21 09:22:34.708 7 WARNING nova.virt.block_device [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Detaching volume from unknown instance<br>2023-08-21 09:22:34.710 7 WARNING nova.virt.libvirt.driver [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] During detach_volume, instance disappeared.: nova.exception.InstanceNotFound: Instance 5520dbd7-8f12-4041-bd8c-65b4b55b0452 could not be found.<br></div><div><br></div><div><br></div><div><br></div><div>Server show :</div><div><br></div><div>+-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+<br>| Field | Value |<br>+-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+<br>| OS-DCF:diskConfig | AUTO |<br>| OS-EXT-AZ:availability_zone | Watson |<br>| OS-EXT-SRV-ATTR:host | <a href="http://overcloud-novacompute-0.bdxworld.com" target="_blank">overcloud-novacompute-0.bdxworld.com</a> |<br>| OS-EXT-SRV-ATTR:hostname | hkg2-cctv-archive-server-2 |<br>| OS-EXT-SRV-ATTR:hypervisor_hostname | <a href="http://overcloud-novacompute-0.bdxworld.com" target="_blank">overcloud-novacompute-0.bdxworld.com</a> |<br>| OS-EXT-SRV-ATTR:instance_name | instance-0000012f |<br>| OS-EXT-SRV-ATTR:kernel_id | |<br>| OS-EXT-SRV-ATTR:launch_index | 0 |<br>| OS-EXT-SRV-ATTR:ramdisk_id | |<br>| OS-EXT-SRV-ATTR:reservation_id | r-mdrfy13i |<br>| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |<br>| OS-EXT-SRV-ATTR:user_data | None |<br>| OS-EXT-STS:power_state | Running |<br>| OS-EXT-STS:task_state | None |<br>| OS-EXT-STS:vm_state | active |<br>| OS-SRV-USG:launched_at | 2023-01-05T09:49:54.000000 |<br>| OS-SRV-USG:terminated_at | None |<br>| accessIPv4 | |<br>| accessIPv6 | |<br>| addresses | HKG2-CCTV=192.168.240.14 |<br>| config_drive | True |<br>| created | 2023-01-05T09:49:46Z |<br>| description | HKG2-CCTV-Archive-server-2 |<br>| flavor | disk='150', ephemeral='0', extra_specs.hw:cpu_cores='10', extra_specs.hw:cpu_sockets='2', original_name='CCTV-VM-Flavor', ram='16384', swap='0', |<br>| | vcpus='10' |<br>| hostId | 570b5ef66c44d3f80a1a498c9dbd8b9a2b690a1aee748bc9cb2cf6fb |<br>| host_status | UP |<br>| id | 5520dbd7-8f12-4041-bd8c-65b4b55b0452 |<br>| image | N/A (booted from volume) |<br>| key_name | None |<br>| locked | False |<br>| locked_reason | None |<br>| name | HKG2-CCTV-Archive-server-2 |<br>| progress | 0 |<br>| project_id | 265571cce0b54251b17ace88630b5e46 |<br>| properties | |<br>| security_groups | name='CCTV services Incoming' |<br>| | name='CCTV services Outgoing' |<br>| server_groups | [] |<br>| status | ACTIVE |<br>| tags | |<br>| trusted_image_certificates | None |<br>| updated | 2023-08-21T09:22:35Z |<br>| user_id | ff69985eea81fe6f11e9caa3f242623cbddc7ce21cdd864c9585d793533acb9e |<br>| volumes_attached | delete_on_termination='False', id='a005a015-0bb2-498e-9068-ac14570a5f2e' |<br>| | delete_on_termination='False', id='01b9b430-9cfd-43b1-b674-b6fb830ee511' |<br>| | delete_on_termination='False', id='a385a416-1d37-4111-b7ae-f182a54aeaef' |<br>| | delete_on_termination='False', id='95bafa69-b7cc-4d35-9002-723ca556a570' |<br>| | delete_on_termination='False', id='b5091e83-2afb-457c-abf8-ed855156b59f' |<br>| | delete_on_termination='False', id='0e9762bd-5229-433d-825e-6c7a77c6927c' |<br>| | delete_on_termination='False', id='9a00e144-8755-48d5-8907-7fa499172371' |<br>+-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+<br></div><div><br></div><div>Please suggest how I can migrate the instance.</div><div><br></div><div>With regards,</div><div>Swogat Pradhan</div></div>
</blockquote></div>