2021-04-19 08:48:15.983 2886 WARNING oslo.service.loopingcall [-] Function 'nova.servicegroup.drivers.db.DbDriver._report_state' run outlasted interval by 3.34 sec
2021-04-19 08:48:16.280 2886 INFO nova.compute.claims [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Claim successful on node s700066.463.os.mcgown.enterprises
2021-04-19 08:48:16.397 2886 INFO nova.compute.resource_tracker [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Updating resource usage from migration 3316d958-0a63-47d8-b1f6-a9e28a49ab82
2021-04-19 08:48:16.786 2886 INFO nova.compute.manager [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Migrating
2021-04-19 08:48:24.852 2886 INFO nova.network.neutron [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Updating port 2e7d818a-43e1-48fb-a4d3-9e36034a46bf with attributes {'binding:host_id': 's700066.463.os.mcgown.enterprises', 'device_owner': 'compute:az-elcom-1'}
2021-04-19 08:48:30.682 2886 INFO nova.compute.resource_tracker [req-d953451e-644d-4615-b946-b52486e28348 - - - - -] [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Updating resource usage from migration 3316d958-0a63-47d8-b1f6-a9e28a49ab82
2021-04-19 08:48:31.894 2886 INFO oslo.privsep.daemon [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/nova/nova-dist.conf', '--config-file', '/etc/nova/nova.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmps2aeicp8/privsep.sock']
2021-04-19 08:48:32.492 2886 INFO oslo.privsep.daemon [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] Spawned new privsep daemon via rootwrap
2021-04-19 08:48:32.432 961072 INFO oslo.privsep.daemon [-] privsep daemon starting
2021-04-19 08:48:32.434 961072 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
2021-04-19 08:48:32.436 961072 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
2021-04-19 08:48:32.436 961072 INFO oslo.privsep.daemon [-] privsep daemon running as pid 961072
2021-04-19 08:48:36.048 2886 INFO nova.virt.libvirt.driver [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Creating image
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Setting instance vm_state to ERROR: nova.exception.InternalError: Unexpected vif_type=binding_failed
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Traceback (most recent call last):
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 10086, in _error_out_instance_on_exception
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     yield
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5850, in _finish_resize_helper
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     request_spec)
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5786, in _finish_resize
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     self._set_instance_info(instance, old_flavor)
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     self.force_reraise()
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     six.reraise(self.type_, self.value, self.tb)
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     raise value
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5774, in _finish_resize
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     block_device_info, power_on)
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 10576, in finish_migration
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     mdevs=mdevs)
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6501, in _get_guest_xml
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     context, mdevs, accel_info)
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6152, in _get_guest_config
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     instance, vif, image_meta, flavor, virt_type,
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/vif.py", line 583, in get_config
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]     raise exception.InternalError(_('Unexpected vif_type=%s') % vif_type)
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] nova.exception.InternalError: Unexpected vif_type=binding_failed
2021-04-19 08:48:36.119 2886 ERROR nova.compute.manager [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3]
2021-04-19 08:48:36.350 2886 INFO nova.compute.manager [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Deleting allocations for old flavor on source node s700030.463.os.mcgown.enterprises after finish_resize failure. You may be able to recover the instance by hard rebooting it.
2021-04-19 08:48:36.456 2886 INFO nova.scheduler.client.report [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] Deleted allocation for migration 3316d958-0a63-47d8-b1f6-a9e28a49ab82
2021-04-19 08:48:37.152 2886 INFO nova.compute.manager [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] [instance: 64229d87-4cbb-44d1-ba8a-5fe63c9c40f3] Successfully reverted task state from resize_finish on failure for instance.
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server [req-228b5f98-e3a4-4c22-8c90-eacce6efb091 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] Exception during message handling: nova.exception.InternalError: Unexpected vif_type=binding_failed
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 78, in wrapped
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     self.force_reraise()
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     raise value
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 69, in wrapped
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 189, in decorated_function
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     "Error: %s", e, instance=instance)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     self.force_reraise()
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     raise value
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 159, in decorated_function
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/utils.py", line 1434, in decorated_function
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 147, in decorated_function
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 217, in decorated_function
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     self.force_reraise()
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     raise value
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 205, in decorated_function
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5834, in finish_resize
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     context, instance, migration)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     self.force_reraise()
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     raise value
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5816, in finish_resize
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     migration, request_spec)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5850, in _finish_resize_helper
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     request_spec)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5786, in _finish_resize
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     self._set_instance_info(instance, old_flavor)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     self.force_reraise()
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     raise value
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5774, in _finish_resize
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     block_device_info, power_on)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 10576, in finish_migration
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     mdevs=mdevs)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6501, in _get_guest_xml
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     context, mdevs, accel_info)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6152, in _get_guest_config
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     instance, vif, image_meta, flavor, virt_type,
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/vif.py", line 583, in get_config
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server     raise exception.InternalError(_('Unexpected vif_type=%s') % vif_type)
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server nova.exception.InternalError: Unexpected vif_type=binding_failed
2021-04-19 08:48:37.164 2886 ERROR oslo_messaging.rpc.server