Open Stack

Hello,

Did you check /var/log/libvirt/libvirtd.log?

Cheers,
-- 
Arnaud Morin

On 03.09.20 - 10:59, Grant Morley wrote:
> Hi All,
> 
> I was wondering if anyone has come across an issue with libvirt seemingly
> having an issue with instances all of a sudden "locking up" with the
> following error:
> 
> Failed to terminate process 2263874 with SIGKILL: Device or resource busy
> 
> In the nova logs I am seeing:
> 
> 2020-09-03 09:13:43.208 2659995 INFO nova.compute.manager
> [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d
> 8cce5e1532e6435a90b168077664bbdf - default default] [instance:
> f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Rebooting instance
> 2020-09-03 09:15:52.429 2659995 WARNING nova.virt.libvirt.driver
> [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d
> 8cce5e1532e6435a90b168077664bbdf - default default] [instance:
> f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Failed to soft reboot instance. Trying
> hard reboot.
> 2020-09-03 09:16:32.450 2659995 WARNING nova.virt.libvirt.driver
> [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d
> 8cce5e1532e6435a90b168077664bbdf - default default] [instance:
> f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Error from libvirt during destroy.
> Code=38 Error=Failed to terminate process 2263874 with SIGKILL: Device or
> resource busy; attempt 1 of 3: libvirtError: Failed to terminate process
> 2263874 with SIGKILL: Device or resource busy
> 2020-09-03 09:17:12.484 2659995 WARNING nova.virt.libvirt.driver
> [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d
> 8cce5e1532e6435a90b168077664bbdf - default default] [instance:
> f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Error from libvirt during destroy.
> Code=38 Error=Failed to terminate process 2263874 with SIGKILL: Device or
> resource busy; attempt 2 of 3: libvirtError: Failed to terminate process
> 2263874 with SIGKILL: Device or resource busy
> 2020-09-03 09:17:52.516 2659995 WARNING nova.virt.libvirt.driver
> [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d
> 8cce5e1532e6435a90b168077664bbdf - default default] [instance:
> f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Error from libvirt during destroy.
> Code=38 Error=Failed to terminate process 2263874 with SIGKILL: Device or
> resource busy; attempt 3 of 3: libvirtError: Failed to terminate process
> 2263874 with SIGKILL: Device or resource busy
> 2020-09-03 09:17:52.526 2659995 ERROR nova.compute.manager
> [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d
> 8cce5e1532e6435a90b168077664bbdf - default default] [instance:
> f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Cannot reboot instance: Failed to
> terminate process 2263874 with SIGKILL: Device or resource busy:
> libvirtError: Failed to terminate process 2263874 with SIGKILL: Device or
> resource busy
> 2020-09-03 09:17:53.026 2659995 INFO nova.compute.manager
> [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d
> 8cce5e1532e6435a90b168077664bbdf - default default] [instance:
> f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Successfully reverted task state from
> reboot_started on failure for instance.
> 
> It seems to be caused when a reboot happens to an instance.
> 
> If you reset the state and try again, the same error occurs.  You also
> seemingly cannot kill off any libvirt process that is attached to that
> instance.
> 
> To me it looks like it could be a kernel issue with libvirt but I could be
> wrong?
> 
> Does anyone know of a workaround for this other than maybe restarting a
> compute host?
> 
> Many thanks,
> 
> 
>