[Ussuri] Auto shutdown VM

Pierre Riteau pierre at stackhpc.com
Mon Jan 4 10:14:47 UTC 2021


Hi Deepa,

You mention checking dmesg *inside* the VM. But have you checked dmesg
on the hypervisor? It's possible your qemu-kvm processes are
terminated by the kernel out-of-memory (OOM) killer because they try
to allocate more memory than available.

Best wishes,
Pierre Riteau (priteau)


On Wed, 18 Nov 2020 at 03:44, Deepa KR <deepa.kr at fingent.com> wrote:
>
> Hello Mohammed
>
> Thanks for the response.
> No error message inside vm. Have checked dmesg, syslog etc .
>
> I mentioned vm is shutting down itself because of error messages Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4 from hypervisor.
>
> Sent from my iPhone
>
> > On 17-Nov-2020, at 11:35 PM, Mohammed Naser <mnaser at vexxhost.com> wrote:
> >
> > On Tue, Nov 17, 2020 at 12:46 PM Deepa KR <deepa.kr at fingent.com> wrote:
> >>
> >>  Hi All
> >>
> >> We have a Openstack setup with the Ussuri Version and I am regularly facing auto shutdown of a few VMs (ubuntu16.04) randomly .
> >> If I restart then the instance is back .
> >>
> >> From logs I was able to see the messages below .
> >>
> >> WARNING nova.compute.manager [req-2a21d455-ac04-44aa-b248-4776e5109013 813f3fb52c434e38991bb90aa4771541 10b5279cb6f64ca19871f132a2cee1a3 - default default] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Received unexpected event network-vif-unplugged-e97839a1-bbc4-4d26-af30-768ca3630ce9 for instance with vm_state active and task_state None.
> >> INFO nova.compute.manager [-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] VM Stopped (Lifecycle Event)
> >> INFO nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
> >> syslog:Nov 13 07:01:07 fgshwbucehyp04 nova-compute[2680204]: 2020-11-13 07:01:07.684 2680204 WARNING nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
> >>  nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance is already powered off in the hypervisor when stop is called.
> >> nova.virt.libvirt.driver [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance already shutdown.
> >> nova.virt.libvirt.driver [-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully.
> >> nova.compute.manager [req-7a0a0d03-e286-42f0-9e36-38a432f236f3 d9ca03b9d0884d51a26a39b6c82f02eb 304d859c43df4de4944ca5623f7f455c - default default] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Get console output
> >> nova.virt.libvirt.driver [-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully.
> >>
> >> I searched a few blogs and forums but couldn't find a solution to it .
> >>
> >> Few mentioned to add sync_power_state_interval=-1 in /etc/nova/nova.conf .But understood that this will help only when nova stops vm.
> >> But in this case vm itself is shutting down (Instance shutdown by itself. Calling the stop API)
> >> Also no memory issue in VM nor the hypervisor.
> >> Also did apt-get upgrade .
> >>
> >> It would be great if anyone can shed light to this issue.
> >
> > You should check and see if there is anything inside `dmesg` that
> > shows the VM dying (any segfaults?).  Also, it's possible that the VM
> > itself is shutting off so maybe you should check ni its logs.
> >
> >> Regards,
> >> Deepa K R
> >>
> >> Sent from my iPhone
> >
> >
> >
> > --
> > Mohammed Naser
> > VEXXHOST, Inc.
>



More information about the openstack-discuss mailing list