On Tue, Nov 17, 2020 at 9:43 PM Deepa KR deepa.kr@fingent.com wrote:
Hello Mohammed
Thanks for the response. No error message inside vm. Have checked dmesg, syslog etc .
I mentioned vm is shutting down itself because of error messages Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4 from hypervisor.
This really doesn’t mean that the VM is shutting itself down. This just means that Nova has noticed that the power state of the VM doesn’t match the database one.
You would also see this error message in any scenario where the VM disappeared from libvirt
Sent from my iPhone
On 17-Nov-2020, at 11:35 PM, Mohammed Naser mnaser@vexxhost.com wrote:
On Tue, Nov 17, 2020 at 12:46 PM Deepa KR deepa.kr@fingent.com wrote:
Hi All
We have a Openstack setup with the Ussuri Version and I am regularly
facing auto shutdown of a few VMs (ubuntu16.04) randomly .
If I restart then the instance is back .
From logs I was able to see the messages below .
WARNING nova.compute.manager [req-2a21d455-ac04-44aa-b248-4776e5109013
813f3fb52c434e38991bb90aa4771541 10b5279cb6f64ca19871f132a2cee1a3 - default default] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Received unexpected event network-vif-unplugged-e97839a1-bbc4-4d26-af30-768ca3630ce9 for instance with vm_state active and task_state None.
INFO nova.compute.manager [-] [instance:
28cd861c-ef15-444a-a902-9cac643c72b5] VM Stopped (Lifecycle Event)
INFO nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - -
- -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] During
_sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
syslog:Nov 13 07:01:07 fgshwbucehyp04 nova-compute[2680204]: 2020-11-13
07:01:07.684 2680204 WARNING nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - -
-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance is already powered off in the hypervisor when stop is called.
nova.virt.libvirt.driver [req-8261f607-4f1e-459d-85d4-e269694dd477 - -
- -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance already
shutdown.
nova.virt.libvirt.driver [-] [instance:
28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully.
nova.compute.manager [req-7a0a0d03-e286-42f0-9e36-38a432f236f3
d9ca03b9d0884d51a26a39b6c82f02eb 304d859c43df4de4944ca5623f7f455c - default default] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Get console output
nova.virt.libvirt.driver [-] [instance:
28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully.
I searched a few blogs and forums but couldn't find a solution to it .
Few mentioned to add sync_power_state_interval=-1 in
/etc/nova/nova.conf .But understood that this will help only when nova stops vm.
But in this case vm itself is shutting down (Instance shutdown by
itself. Calling the stop API)
Also no memory issue in VM nor the hypervisor. Also did apt-get upgrade .
It would be great if anyone can shed light to this issue.
You should check and see if there is anything inside `dmesg` that shows the VM dying (any segfaults?). Also, it's possible that the VM itself is shutting off so maybe you should check ni its logs.
Regards, Deepa K R
Sent from my iPhone
-- Mohammed Naser VEXXHOST, Inc.