5 Jan
2021
5 Jan
'21
7:25 a.m.
Hello Pierre Yeah have checked hypervisor too ..No error related to kernel out-of-memory (OOM) in hypervisor On Mon, Jan 4, 2021 at 3:45 PM Pierre Riteau <pierre@stackhpc.com> wrote: > Hi Deepa, > > You mention checking dmesg *inside* the VM. But have you checked dmesg > on the hypervisor? It's possible your qemu-kvm processes are > terminated by the kernel out-of-memory (OOM) killer because they try > to allocate more memory than available. > > Best wishes, > Pierre Riteau (priteau) > > > On Wed, 18 Nov 2020 at 03:44, Deepa KR <deepa.kr@fingent.com> wrote: > > > > Hello Mohammed > > > > Thanks for the response. > > No error message inside vm. Have checked dmesg, syslog etc . > > > > I mentioned vm is shutting down itself because of error messages > Instance shutdown by itself. Calling the stop API. Current vm_state: > active, current task_state: None, original DB power_state: 1, current VM > power_state: 4 from hypervisor. > > > > Sent from my iPhone > > > > > On 17-Nov-2020, at 11:35 PM, Mohammed Naser <mnaser@vexxhost.com> > wrote: > > > > > > On Tue, Nov 17, 2020 at 12:46 PM Deepa KR <deepa.kr@fingent.com> > wrote: > > >> > > >> Hi All > > >> > > >> We have a Openstack setup with the Ussuri Version and I am regularly > facing auto shutdown of a few VMs (ubuntu16.04) randomly . > > >> If I restart then the instance is back . > > >> > > >> From logs I was able to see the messages below . > > >> > > >> WARNING nova.compute.manager > [req-2a21d455-ac04-44aa-b248-4776e5109013 813f3fb52c434e38991bb90aa4771541 > 10b5279cb6f64ca19871f132a2cee1a3 - default default] [instance: > 28cd861c-ef15-444a-a902-9cac643c72b5] Received unexpected event > network-vif-unplugged-e97839a1-bbc4-4d26-af30-768ca3630ce9 for instance > with vm_state active and task_state None. > > >> INFO nova.compute.manager [-] [instance: > 28cd861c-ef15-444a-a902-9cac643c72b5] VM Stopped (Lifecycle Event) > > >> INFO nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - > - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] During > _sync_instance_power_state the DB power_state (1) does not match the > vm_power_state from the hypervisor (4). Updating power_state in the DB to > match the hypervisor. > > >> syslog:Nov 13 07:01:07 fgshwbucehyp04 nova-compute[2680204]: > 2020-11-13 07:01:07.684 2680204 WARNING nova.compute.manager > [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: > 28cd861c-ef15-444a-a902-9cac643c72b5] Instance shutdown by itself. Calling > the stop API. Current vm_state: active, current task_state: None, original > DB power_state: 1, current VM power_state: 4 > > >> nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - > - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance is already > powered off in the hypervisor when stop is called. > > >> nova.virt.libvirt.driver [req-8261f607-4f1e-459d-85d4-e269694dd477 - > - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance already > shutdown. > > >> nova.virt.libvirt.driver [-] [instance: > 28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully. > > >> nova.compute.manager [req-7a0a0d03-e286-42f0-9e36-38a432f236f3 > d9ca03b9d0884d51a26a39b6c82f02eb 304d859c43df4de4944ca5623f7f455c - default > default] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Get console output > > >> nova.virt.libvirt.driver [-] [instance: > 28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully. > > >> > > >> I searched a few blogs and forums but couldn't find a solution to it . > > >> > > >> Few mentioned to add sync_power_state_interval=-1 in > /etc/nova/nova.conf .But understood that this will help only when nova > stops vm. > > >> But in this case vm itself is shutting down (Instance shutdown by > itself. Calling the stop API) > > >> Also no memory issue in VM nor the hypervisor. > > >> Also did apt-get upgrade . > > >> > > >> It would be great if anyone can shed light to this issue. > > > > > > You should check and see if there is anything inside `dmesg` that > > > shows the VM dying (any segfaults?). Also, it's possible that the VM > > > itself is shutting off so maybe you should check ni its logs. > > > > > >> Regards, > > >> Deepa K R > > >> > > >> Sent from my iPhone > > > > > > > > > > > > -- > > > Mohammed Naser > > > VEXXHOST, Inc. > > > -- Regards, Deepa K R | DevOps Team Lead USA | UAE | INDIA | AUSTRALIA
