<div dir="ltr">Thank you Mike! I see that setting <span style="color:rgb(0,0,0);white-space:pre-wrap">mem_stats_period_seconds = 0 in nova.conf libvirt section is mentioned in the bug ticket. This has also been mentioned as a workaround by Corbin. Is this something that you have tested? This may be the least intrusive workaround. Thx will</span></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 28, 2016 at 11:15 PM, Mike Smith <span dir="ltr"><<a href="mailto:mismith@overstock.com" target="_blank">mismith@overstock.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
William -
<div><br>
</div>
<div>That is probably what you have hit then. There are several bug tickets out there that are related, I don’t have the specific one I found handy but it’s similar to the one Marcus just posted. I found it by Googling around for the error late one
night.</div>
<div><br>
</div>
<div>In our case, we had most of our hypervisors on the 1.5.3-105.el7_2.4 version but had a couple of new ones that ended up with 1.5.3-105.el7_2.7. Migrating to those new hypervisors resulted in the VMs being shut down, but migrating from the new
to the old worked fine. So for my case I live-migrated all the VMs from the new compute hosts to the old ones, did the yum downgrade of those packages on the new hosts, and then rebooted those hosts for good measure. After that I no longer had the problem.</div>
<div><br>
</div>
<div>Good luck! <br>
<div><span class="">
<div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<br>
Mike Smith<br>
Lead Cloud Systems Architect</div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<a href="http://overstock.com" target="_blank">Overstock.com</a><br>
<br>
<br>
</div>
</div>
</div>
</div>
<br>
</span><div><div class="h5"><div>
<blockquote type="cite">
<div>On Sep 28, 2016, at 8:55 AM, William Josefsson <<a href="mailto:william.josefson@gmail.com" target="_blank">william.josefson@gmail.com</a>> wrote:</div>
<br>
<div>
<div dir="ltr">thanks Mike! yes, I checked and I've got the versions you mentioned installed:
<div><br>
</div>
<div>
<div>yum list installed | grep "qemu-"</div>
<div>qemu-img.x86_64 10:1.5.3-105.el7_2.7 @updates</div>
<div>qemu-kvm.x86_64 10:1.5.3-105.el7_2.7 @updates</div>
<div>qemu-kvm-common.x86_64 10:1.5.3-105.el7_2.7 @updates</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Is there any reference or bugzilla ticket for this issue, or how did you track it down?</div>
<div><br>
</div>
<div>Can you also please advice on the command: "yum downgrade qemu-kvm qemu-kvm-common qemu-img” as I'm having running VMs in my environment, do I need to restart libvirtd after this, and would it affect any of my instances? thx will</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Sep 28, 2016 at 10:47 PM, Mike Smith <span dir="ltr">
<<a href="mailto:mismith@overstock.com" target="_blank">mismith@overstock.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">There is a bug in the following:
<div><br>
</div>
<div>
<div>
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<div style="text-align:start;text-indent:0px;word-wrap:break-word">qemu-kvm-1.5.3-105.el7_2.7</div>
<div style="text-align:start;text-indent:0px;word-wrap:break-word">qemu-img-1.5.3-105.el7_2.7</div>
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<div style="word-wrap:break-word">qemu-kvm-common-1.5.3-105.el7_<wbr>2.7</div>
<div><br>
</div>
<div>Are you running these versions? We encountered this same issue and fixed it by running "yum downgrade qemu-kvm qemu-kvm-common qemu-img” which took us back to the el7_2.4</div>
<div><br>
</div>
</div>
<div style="text-align:start;text-indent:0px;word-wrap:break-word"><br>
</div>
<div style="text-align:start;text-indent:0px;word-wrap:break-word">Mike Smith<br>
Lead Cloud Systems Architect</div>
<div style="letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-align:start;text-indent:0px;word-wrap:break-word">
<a href="http://overstock.com/" target="_blank">Overstock.com</a><br>
<br>
<br>
</div>
</div>
</div>
</div>
<br>
<div>
<blockquote type="cite">
<div>
<div>
<div>On Sep 28, 2016, at 8:30 AM, William Josefsson <<a href="mailto:william.josefson@gmail.com" target="_blank">william.josefson@gmail.com</a>> wrote:</div>
<br>
</div>
</div>
<div>
<div>
<div>
<div dir="ltr">Hi,<br>
<br>
I have problems with nova live-migration for my CentOS7 and RHEL7 VMs. The live migrations works fine for Windows2012R2 and Ubuntu1404 VMs.<br>
<br>
For CentOS7/RHEL based images I get this error in the Destination node in nova-compute.log. Also console for logged in VM users on the VM freeze.
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>2016-09-28 10:49:24.101 353935 INFO nova.compute.manager [instance: cd0b605d] VM Resumed (Lifecycle Event)<br>
2016-09-28 10:49:24.339 353935 INFO nova.compute.manager [instance: cd0b605d] VM Resumed (Lifecycle Event)<br>
2016-09-28 10:49:25.866 353935 INFO nova.compute.manager [instance: cd0b605d] Post operation of migration started<br>
2016-09-28 10:49:39.410 353935 INFO nova.compute.manager [instance: cd0b605d] VM Stopped (Lifecycle Event)<br>
<b>2016-09-28 10:49:39.532 353935 WARNING nova.compute.manager [instance: cd0b605d] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 4, current VM power_state: 4</b><br>
2016-09-28 10:49:39.668 353935 INFO nova.compute.manager [instance: cd0b605d] Instance is already powered off in the hypervisor when stop is called.<br>
2016-09-28 10:49:39.736 353935 INFO nova.virt.libvirt.driver [instance: cd0b605d] Instance already shutdown.<br>
2016-09-28 10:49:39.743 353935 INFO nova.virt.libvirt.driver [instance: cd0b605d] Instance destroyed successfully.</div>
<div><br>
</div>
<div><br>
</div>
<div>Eventually the VM ends up in SHUTDOWN state on the destination node. </div>
<div><br>
</div>
<div>I'm on CentOS7/Liberty, and my storage backend is CEPH(Hammer 0.94.9).<br>
</div>
<div><br>
</div>
<div>Please advice. thx will</div>
</div>
</div>
</div>
______________________________<wbr>_________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.open<wbr>stack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-operators</a><br>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<hr>
<font face="Arial" color="Gray" size="1"><br>
CONFIDENTIALITY NOTICE: This message is intended only for the use and review of the individual or entity to which it is addressed and may contain information that is privileged and confidential. If the reader of this message is not the intended recipient, or
the employee or agent responsible for delivering the message solely to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in
error, please notify sender immediately by telephone or return email. Thank you.<br>
</font></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div></div></div>
</div><div><div class="h5">
<br>
<hr>
<font face="Arial" color="Gray" size="1"><br>
CONFIDENTIALITY NOTICE: This message is intended only for the use and review of the individual or entity to which it is addressed and may contain information that is privileged and confidential. If the reader of this message is not the intended recipient, or
the employee or agent responsible for delivering the message solely to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in
error, please notify sender immediately by telephone or return email. Thank you.<br>
</font>
</div></div></div>
</blockquote></div><br></div>