qemu ussuri can't launch instance
Hello,
Does anyone know where to find the problem? Searched few days in google :)
2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [req-f16a5b6c-3475-4322-a664-45f3bd9c041a cb4c181d25a14dda958e40792968f97e 5242bcab73714ab192cca8d909b72e9a - default default] [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] Instance failed to spawn: libvirt.libvirtError: internal error: cannot load AppArmor$ 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] Traceback (most recent call last): 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2614, in _build_resources 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] yield resources 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2378, in _build_and_run_instance 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] accel_info=accel_info) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3580, in spawn 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] power_on=power_on) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6438, in _create_domain_and_network 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] destroy_disks_on_failure) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] self.force_reraise() 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] six.reraise(self.type_, self.value, self.tb) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] raise value 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6410, in _create_domain_and_network 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] post_xml_callback=post_xml_callback) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6351, in _create_domain 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] guest.launch(pause=pause) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 142, in launch 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] self._encoded_xml, errors='ignore') 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] self.force_reraise() 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] six.reraise(self.type_, self.value, self.tb) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] raise value 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 137, in launch 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] return self._domain.createWithFlags(flags) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 193, in doit 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] result = proxy_call(self._autowrap, f, *args, **kwargs) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 151, in proxy_call 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] rv = execute(f, *args, **kwargs) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 132, in execute 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] six.reraise(c, e, tb) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] raise value 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 86, in tworker 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] rv = meth(*args, **kwargs) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/libvirt.py", line 1265, in createWithFlags 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] libvirt.libvirtError: internal error: cannot load AppArmor profile 'libvirt-63bb4205-fcbb-4f2b-9ade-e055e7ff966f' 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f]
On Mon, 2020-10-05 at 23:48 +0300, ops@clustspace.com wrote:
Hello,
Does anyone know where to find the problem? Searched few days in google :)
this looks like the trace this looks like a packaging issue ibvirt.libvirtError: internal error: cannot load AppArmor profile
specificly it looks like there is a conflict between your installation and apparmor. this does not looks to intially be a nova/openstack or libvirt/qemu issue outside of a possible misinstallation or misconfiguration of those services to be in conflict with the apparmor profile.
the dmesg or audit log may provie insite into exactuly whic calls are being blocked and how to adress that.
2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [req-f16a5b6c-3475-4322-a664-45f3bd9c041a cb4c181d25a14dda958e40792968f97e 5242bcab73714ab192cca8d909b72e9a - default default] [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] Instance failed to spawn: libvirt.libvirtError: internal error: cannot load AppArmor$ 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] Traceback (most recent call last): 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2614, in _build_resources 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] yield resources 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2378, in _build_and_run_instance 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] accel_info=accel_info) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3580, in spawn 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] power_on=power_on) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6438, in _create_domain_and_network 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] destroy_disks_on_failure) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] self.force_reraise() 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] six.reraise(self.type_, self.value, self.tb) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] raise value 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6410, in _create_domain_and_network 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] post_xml_callback=post_xml_callback) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6351, in _create_domain 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] guest.launch(pause=pause) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 142, in launch 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] self._encoded_xml, errors='ignore') 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] self.force_reraise() 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] six.reraise(self.type_, self.value, self.tb) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] raise value 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 137, in launch 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] return self._domain.createWithFlags(flags) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 193, in doit 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] result = proxy_call(self._autowrap, f, *args, **kwargs) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 151, in proxy_call 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] rv = execute(f, *args, **kwargs) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 132, in execute 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] six.reraise(c, e, tb) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] raise value 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 86, in tworker 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] rv = meth(*args, **kwargs) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] File "/usr/lib/python3/dist-packages/libvirt.py", line 1265, in createWithFlags 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f] libvirt.libvirtError: internal error: cannot load AppArmor profile 'libvirt-63bb4205-fcbb-4f2b-9ade-e055e7ff966f' 2020-10-05 23:11:04.054 1585 ERROR nova.compute.manager [instance: 63bb4205-fcbb-4f2b-9ade-e055e7ff966f]
On 10/5/20 11:07 PM, Sean Mooney wrote:
On Mon, 2020-10-05 at 23:48 +0300, ops@clustspace.com wrote:
Hello,
Does anyone know where to find the problem? Searched few days in google :)
this looks like the trace this looks like a packaging issue ibvirt.libvirtError: internal error: cannot load AppArmor profile
specificly it looks like there is a conflict between your installation and apparmor. this does not looks to intially be a nova/openstack or libvirt/qemu issue outside of a possible misinstallation or misconfiguration of those services to be in conflict with the apparmor profile.
the dmesg or audit log may provie insite into exactuly whic calls are being blocked and how to adress that.
With OpenStack, /etc/libvirt/qemu.conf must be configured with:
security_driver = "apparmor"
for Buster, and the same line commented-out for Stertch. I'm not sure what distro we're talking about here...
I hope this helps, Cheers,
Thomas Goirand (zigo)
participants (3)
-
ops＠clustspace.com
-
Sean Mooney
-
Thomas Goirand