[Openstack] live_migration only using 8 Mb speed

Matthew Thode prometheanfire at gentoo.org
Thu Aug 23 06:30:32 UTC 2018


On 18-08-22 23:04:57, Satish Patel wrote:
> Mathew,
> 
> I haven't applied any patch yet but i am noticing in cluster some host
> migrating VM super fast and some host migrating very slow. Is this
> known behavior?
> 
> On Wed, Aug 22, 2018 at 11:28 AM, Matthew Thode
> <prometheanfire at gentoo.org> wrote:
> > On 18-08-22 10:58:48, Satish Patel wrote:
> >> Matthew,
> >>
> >> I have two option looks like, correct me if i am wrong.
> >>
> >> 1. I have two option, upgrade minor release from 17.0.7-6-g9187bb1 to
> >> 17.0.8-23-g0aff517  and upgrade full OSA
> >>
> >> 2. Just do override as you said "nova_git_install_branch:" in my
> >> /etc/openstack_deploy/user_variables.yml  file, and run playbooks.
> >>
> >>
> >> I think option [2] is safe to just touch specific component, also am i
> >> correct about override in /etc/openstack_deploy/user_variables.yml
> >> file?
> >>
> >> You mentioned "nova_git_install_branch:
> >> dee99b1ed03de4b6ded94f3cf6d2ea7214bca93b" but i believe it should be
> >> "a9c9285a5a68ab89a6543d143c364d90a01cd51c" am i correct?
> >>
> >>
> >>
> >> On Wed, Aug 22, 2018 at 10:46 AM, Matthew Thode
> >> <prometheanfire at gentoo.org> wrote:
> >> > On 18-08-22 10:33:11, Satish Patel wrote:
> >> >> Thanks Matthew,
> >> >>
> >> >> Can i put that sha in my OSA at
> >> >> playbooks/defaults/repo_packages/openstack_services.yml by hand and
> >> >> run playbooks [repo/nova] ?
> >> >>
> >> >> On Wed, Aug 22, 2018 at 10:24 AM, Matthew Thode
> >> >> <prometheanfire at gentoo.org> wrote:
> >> >> > On 18-08-22 08:35:09, Satish Patel wrote:
> >> >> >> Currently in stable/queens i am seeing this sha
> >> >> >> https://github.com/openstack/openstack-ansible/blob/stable/queens/ansible-role-requirements.yml#L112
> >> >> >>
> >> >> >> On Wed, Aug 22, 2018 at 2:02 AM, Matthew Thode
> >> >> >> <prometheanfire at gentoo.org> wrote:
> >> >> >> > On 18-08-22 01:57:17, Satish Patel wrote:
> >> >> >> >> What I need to upgrade, any specific component?
> >> >> >> >>
> >> >> >> >> I have deployed openstack-ansible
> >> >> >> >>
> >> >> >> >> Sent from my iPhone
> >> >> >> >>
> >> >> >> >> > On Aug 22, 2018, at 1:06 AM, Matthew Thode <prometheanfire at gentoo.org> wrote:
> >> >> >> >> >
> >> >> >> >> >> On 18-08-22 01:02:53, Satish Patel wrote:
> >> >> >> >> >> Matthew,
> >> >> >> >> >>
> >> >> >> >> >> Thanks for reply, Look like i don't have this patch
> >> >> >> >> >> https://review.openstack.org/#/c/591761/
> >> >> >> >> >>
> >> >> >> >> >> So i have to patch following 3 file manually?
> >> >> >> >> >>
> >> >> >> >> >> nova/tests/unit/virt/libvirt/test_driver.py213
> >> >> >> >> >> nova/tests/unit/virt/test_virt_drivers.py2
> >> >> >> >> >> nova/virt/libvirt/driver.py
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >> On Wed, Aug 22, 2018 at 12:42 AM, Matthew Thode
> >> >> >> >> >> <prometheanfire at gentoo.org> wrote:
> >> >> >> >> >>> On 18-08-22 00:27:08, Satish Patel wrote:
> >> >> >> >> >>>> Folks,
> >> >> >> >> >>>>
> >> >> >> >> >>>> I am running openstack queens and hypervisor is kvm, my live migration
> >> >> >> >> >>>> working fine. but somehow it stuck to 8 Mb network speed and taking
> >> >> >> >> >>>> long time to migrate 1G instance.  I have 10Gbps network and i have
> >> >> >> >> >>>> tried to copy 10G file between two compute node and it did copy in 2
> >> >> >> >> >>>> minute, so i am not seeing any network issue also.
> >> >> >> >> >>>>
> >> >> >> >> >>>> it seem live_migration has some bandwidth limit, I have tried
> >> >> >> >> >>>> following option in nova.conf but it didn't work
> >> >> >> >> >>>>
> >> >> >> >> >>>> live_migration_bandwidth = 500
> >> >> >> >> >>>>
> >> >> >> >> >>>> My nova.conf look like following:
> >> >> >> >> >>>>
> >> >> >> >> >>>> live_migration_uri =
> >> >> >> >> >>>> "qemu+ssh://nova@%s/system?no_verify=1&keyfile=/var/lib/nova/.ssh/id_rsa"
> >> >> >> >> >>>> live_migration_tunnelled = True
> >> >> >> >> >>>> live_migration_bandwidth = 500
> >> >> >> >> >>>> hw_disk_discard = unmap
> >> >> >> >> >>>> disk_cachemodes = network=writeback
> >> >> >> >> >>>>
> >> >> >> >> >>>
> >> >> >> >> >>> Do you have a this patch (and a couple of patches up to it)?
> >> >> >> >> >>> https://bugs.launchpad.net/nova/+bug/1786346
> >> >> >> >> >>>
> >> >> >> >> >
> >> >> >> >> > I don't know if that would cleanly apply (there are other patches that
> >> >> >> >> > changed those functions within the last month and a half.  It'd be best
> >> >> >> >> > to upgrade and not do just one patch (which would be an untested
> >> >> >> >> > process).
> >> >> >> >> >
> >> >> >> >
> >> >> >> > The sha for nova has not been updated yet (next update is 24-48 hours
> >> >> >> > away iirc), once that's done you can use the head of stable/queens from
> >> >> >> > OSA and run a inter-series upgrade (but the minimal thing to do would be
> >> >> >> > to run repo-build and os-nova plays).  I'm not sure when that sha bump
> >> >> >> > will be tagged in a full release if you would rather wait on that.
> >> >> >
> >> >> > it's this sha that needs updating.
> >> >> > https://github.com/openstack/openstack-ansible/blob/stable/queens/playbooks/defaults/repo_packages/openstack_services.yml#L173
> >> >> >
> >> >
> >> > I'm not sure how you are doing overrides, but set the following as an
> >> > override, then rerun the repo-build playbook (to rebuild the nova venv)
> >> > then rerun the nova playbook to install it.
> >> >
> >> > nova_git_install_branch: dee99b1ed03de4b6ded94f3cf6d2ea7214bca93b
> >> >
> >
> > The sha I gave was head of the queens branch of openstack/nova.  It's
> > also the commit in that branch that containst the fix.
> >

I don't think that is known behavior, different memory pressure causing
the difference maybe?

-- 
Matthew Thode (prometheanfire)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20180823/5f0c5d14/attachment.sig>


More information about the Openstack mailing list