[Openstack-operators] [nova] Live migration performance tests on 100 compute nodes

Rui Chen chenrui.momo at gmail.com
Wed Jan 4 03:46:37 UTC 2017


And you can use the nova option "live_migration_inbound_addr" [1] to avoid
the impact of live migration at all, but that require additional network
plane.

[1]:
https://github.com/openstack/nova/blob/master/nova/conf/libvirt.py#L243-L258
[2]:
https://blueprints.launchpad.net/nova/+spec/split-network-plane-for-live-migration

2016-12-30 20:56 GMT+08:00 Koniszewski, Pawel <pawel.koniszewski at intel.com>:

> Matt,
>
>
>
> This was a bandwidth issue. Nova kept being connected to the broker, but
> it started to timeout on RPC messages, e.g., we lost some RPC messages
> triggering post live migration steps which are there to update nova DB to
> reflect new host of an instance.
>
>
>
> The good workaround for this issue is to slightly limit bandwidth used for
> live migrations through nova.conf [1], live_migration_bandwidth config
> option in [libvirt] section. By default it is set to 0, so, basically, it
> is unlimited. Also please be aware that we changed default live migration
> configuration in OpenStack Newton and tunneling is now off by default
> (live_migration_tunnelled in [libvirt] section is set to False) due to huge
> performance impact.
>
>
>
> [1] http://docs.openstack.org/newton/config-reference/
> compute/config-options.html
>
>
>
> Kind Regards,
>
> Pawel Koniszewski
>
>
>
> *From:* tadowguy at gmail.com [mailto:tadowguy at gmail.com] *On Behalf Of *Matt
> Fischer
> *Sent:* Friday, December 30, 2016 5:31 AM
> *To:* Koniszewski, Pawel <pawel.koniszewski at intel.com>
> *Cc:* openstack-operators at lists.openstack.org
> *Subject:* Re: [Openstack-operators] [nova] Live migration performance
> tests on 100 compute nodes
>
>
>
> On Wed, Dec 28, 2016 at 6:11 AM, Koniszewski, Pawel <
> pawel.koniszewski at intel.com> wrote:
>
> Hello everyone,
>
> We made a research to see how live migration performance varies between
> different configurations, especially we aimed to test tunneled vs
> non-tunneled live migrations. To test live migration we simulated a case of
> 0-day patching of 100 compute nodes (including reboot) with workloads that
> are close to the real world workloads. All the results were published [1]
> along with environment configuration and how we built test framework. Hope
> you find this useful.
>
> [1] https://01.org/openstack/blogs/pkoniszewski/2016/ossc-
> zero-day-patching
>
> Kind Regards,
> Pawel Koniszewski
>
>
>
> Thanks for the write-up. I'm curious about your RabbitMQ connection
> failures. Was it nova-compute failing to connect? Was it a bandwidth or
> heartbeat issue?
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170104/337ccd80/attachment.html>


More information about the OpenStack-operators mailing list