[Openstack-operators] Nova live-migration failing for RHEL7/CentOS7 VMs

William Josefsson william.josefson at gmail.com
Sun Oct 2 12:21:56 UTC 2016


Many thanks Corbin! I should go ahead test and install
'centos-release-openstack-liberty' followed by
'centos-release-qemu-ev' in my dev environment.

To get snapshot support working with qemu-kvm 10:1.5.3-105.el7_2.7
(offline recommended), I will also on my Compute nodes set in
nova.conf:

[workarounds]
disable_libvirt_livesnapshot = False

On the other hand, do you folks generally use the RHEV-QEMU stack in
your production deployments, or do you see any major benefits doing so
instead of using the default qemu that comes with CentOS7.2? thx will


On Sun, Oct 2, 2016 at 5:54 AM, Corbin Hendrickson
<corbin.hendrickson at endurance.com> wrote:
> Yes, you can find them here
> http://mirror.centos.org/centos/7/virt/x86_64/kvm-common/ And if memory
> serves me well it still does have the live migration issue, and yes live
> snapshots will not work on this version or even the latest 2.6 qemu will not
> work with live snapshots. I don't have the particular thread from the qemu
> mailing list handy but it seemed they were still trying to decide how they
> wanted to do them going forward as of a few months ago. Which is why if you
> are using 1.5.* qemu you will need to set in your nova.conf under the
> section [workarounds]: disable_libvirt_livesnapshot = False    At which
> point if you invoke a nova image-create (aka snapshot) and the instance is
> running it will perform a snapshot. Keep in mind however, the docs do state
> that live snapshots fail intermittently under load. So it is entirely a
> better option to shutdown the guest and perform the nova image-create as
> opposed to doing it live if you can get away with it.
>
> Corbin Hendrickson
> Endurance Cloud Development Lead - Manager
> Cell: 801-400-0464
>
> On Fri, Sep 30, 2016 at 10:02 PM, William Josefsson
> <william.josefson at gmail.com> wrote:
>>
>> Hi Dave, I use CentOS7.2 with the following package as per OpenStack
>> official documentation, 'centos-release-openstack-liberty'. It works
>> just fine, no major issues so far. I haven't tried snapshots so thanks
>> for highlighting the support may not be there.
>>
>> Correct me if I'm mistaken, if I want to use the RHEV stack of KVM
>> Qemu and other related packages, I should install:
>> 'centos-release-qemu-ev' after installing
>> 'centos-release-openstack-liberty'? thx will
>>
>> On Sat, Oct 1, 2016 at 3:04 AM, David Moreau Simard <dms at redhat.com>
>> wrote:
>> > If you are deploying on CentOS (with RDO?), you can enable the CentOS
>> > Virtualization special interest group [1] repository.
>> >
>> > The repository contains qemu-kvm-ev >= 2.3 backported from RHEV.
>> > It is recommended as the qemu-kvm version from base CentOS
>> > repositories is not high enough and lacks some features (things like
>> > snapshots, iirc).
>> >
>> > qemu-kvm >= 2.3 is actually a requirement in RDO >= Newton and we'll
>> > bundle the CentOS virtualization SIG repository in our release
>> > packages.
>> >
>> > [1]: https://wiki.centos.org/SpecialInterestGroup/Virtualization
>> >
>> > David Moreau Simard
>> > Senior Software Engineer | Openstack RDO
>> >
>> > dmsimard = [irc, github, twitter]
>> >
>> >
>> > On Thu, Sep 29, 2016 at 5:00 AM, William Josefsson
>> > <william.josefson at gmail.com> wrote:
>> >> thanks everyone, I verified setting mem_stats_period_seconds = 0 as
>> >> suggested by Corbin in nova.conf libvirt section, and then restarting
>> >> openstack-nova-compute service and it works!
>> >>
>> >> While this seems to be a workable workaround I'm not sure what's the
>> >> plans
>> >> to permanently fix this in CentOS7.2? thx will
>> >>
>> >>
>> >>
>> >> On Wed, Sep 28, 2016 at 11:37 PM, Corbin Hendrickson
>> >> <corbin.hendrickson at endurance.com> wrote:
>> >>>
>> >>> Oh you can read it in the bug thread, but I forgot to mention, if you
>> >>> put
>> >>> in your nova.conf under the libvirt section mem_stats_period_seconds =
>> >>> 0,
>> >>> and restart nova on the destination (although i'd say just do it on
>> >>> both) it
>> >>> will no longer hit the bug. I tested this a couple weeks back with
>> >>> success.
>> >>>
>> >>> Corbin Hendrickson
>> >>> Endurance Cloud Development Lead - Manager
>> >>> Cell: 801-400-0464
>> >>>
>> >>> On Wed, Sep 28, 2016 at 9:34 AM, Corbin Hendrickson
>> >>> <corbin.hendrickson at endurance.com> wrote:
>> >>>>
>> >>>> It unfortunately is affecting virtually all of Redhat's latest
>> >>>> qemu-kvm
>> >>>> packages. The bug that was unintentionally introduced was done so in
>> >>>> response to CVE-2016-5403  Qemu: virtio: unbounded memory allocation
>> >>>> on host
>> >>>> via guest leading to DoS.
>> >>>>
>> >>>> Late in the bug thread, they finally posted to a new bug created for
>> >>>> the
>> >>>> breaking of live migrate via Bug 1371943 - RHSA-2016-1756 breaks
>> >>>> migration
>> >>>> of instances.
>> >>>>
>> >>>> Based off their posts i've been following it's likely going to "hit
>> >>>> the
>> >>>> shelves" when RHEL 7.3 / CentOS 7.3 comes out. It does look like they
>> >>>> are
>> >>>> backporting it to all their versions of RHEL so that's good.
>> >>>>
>> >>>> But yes this does affect 2.3 as well.
>> >>>>
>> >>>> Corbin Hendrickson
>> >>>> Endurance Cloud Development Lead - Manager
>> >>>> Cell: 801-400-0464
>> >>>>
>> >>>> On Wed, Sep 28, 2016 at 9:13 AM, Van Leeuwen, Robert
>> >>>> <rovanleeuwen at ebay.com> wrote:
>> >>>>>
>> >>>>> > There is a bug in the following:
>> >>>>>
>> >>>>> >
>> >>>>>
>> >>>>> > qemu-kvm-1.5.3-105.el7_2.7
>> >>>>>
>> >>>>> > qemu-img-1.5.3-105.el7_2.7
>> >>>>>
>> >>>>> > qemu-kvm-common-1.5.3-105.el7_2.7
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> You might be better of using the RHEV qemu packages
>> >>>>>
>> >>>>> They are more recent (2.3) and have more features compiled into
>> >>>>> them.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Cheers,
>> >>>>>
>> >>>>> Robert van Leeuwen
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> OpenStack-operators mailing list
>> >>>>> OpenStack-operators at lists.openstack.org
>> >>>>>
>> >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >>
>> >> _______________________________________________
>> >> OpenStack-operators mailing list
>> >> OpenStack-operators at lists.openstack.org
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>> >>
>
>



More information about the OpenStack-operators mailing list