[Openstack] live_migration only using 8 Mb speed
Satish Patel
satish.txt at gmail.com
Thu Aug 23 11:18:24 UTC 2018
I'm testing this in lab, no load yet
Sent from my iPhone
> On Aug 23, 2018, at 2:30 AM, Matthew Thode <prometheanfire at gentoo.org> wrote:
>
>> On 18-08-22 23:04:57, Satish Patel wrote:
>> Mathew,
>>
>> I haven't applied any patch yet but i am noticing in cluster some host
>> migrating VM super fast and some host migrating very slow. Is this
>> known behavior?
>>
>> On Wed, Aug 22, 2018 at 11:28 AM, Matthew Thode
>> <prometheanfire at gentoo.org> wrote:
>>> On 18-08-22 10:58:48, Satish Patel wrote:
>>>> Matthew,
>>>>
>>>> I have two option looks like, correct me if i am wrong.
>>>>
>>>> 1. I have two option, upgrade minor release from 17.0.7-6-g9187bb1 to
>>>> 17.0.8-23-g0aff517 and upgrade full OSA
>>>>
>>>> 2. Just do override as you said "nova_git_install_branch:" in my
>>>> /etc/openstack_deploy/user_variables.yml file, and run playbooks.
>>>>
>>>>
>>>> I think option [2] is safe to just touch specific component, also am i
>>>> correct about override in /etc/openstack_deploy/user_variables.yml
>>>> file?
>>>>
>>>> You mentioned "nova_git_install_branch:
>>>> dee99b1ed03de4b6ded94f3cf6d2ea7214bca93b" but i believe it should be
>>>> "a9c9285a5a68ab89a6543d143c364d90a01cd51c" am i correct?
>>>>
>>>>
>>>>
>>>> On Wed, Aug 22, 2018 at 10:46 AM, Matthew Thode
>>>> <prometheanfire at gentoo.org> wrote:
>>>>> On 18-08-22 10:33:11, Satish Patel wrote:
>>>>>> Thanks Matthew,
>>>>>>
>>>>>> Can i put that sha in my OSA at
>>>>>> playbooks/defaults/repo_packages/openstack_services.yml by hand and
>>>>>> run playbooks [repo/nova] ?
>>>>>>
>>>>>> On Wed, Aug 22, 2018 at 10:24 AM, Matthew Thode
>>>>>> <prometheanfire at gentoo.org> wrote:
>>>>>>> On 18-08-22 08:35:09, Satish Patel wrote:
>>>>>>>> Currently in stable/queens i am seeing this sha
>>>>>>>> https://github.com/openstack/openstack-ansible/blob/stable/queens/ansible-role-requirements.yml#L112
>>>>>>>>
>>>>>>>> On Wed, Aug 22, 2018 at 2:02 AM, Matthew Thode
>>>>>>>> <prometheanfire at gentoo.org> wrote:
>>>>>>>>> On 18-08-22 01:57:17, Satish Patel wrote:
>>>>>>>>>> What I need to upgrade, any specific component?
>>>>>>>>>>
>>>>>>>>>> I have deployed openstack-ansible
>>>>>>>>>>
>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>
>>>>>>>>>>>> On Aug 22, 2018, at 1:06 AM, Matthew Thode <prometheanfire at gentoo.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On 18-08-22 01:02:53, Satish Patel wrote:
>>>>>>>>>>>> Matthew,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for reply, Look like i don't have this patch
>>>>>>>>>>>> https://review.openstack.org/#/c/591761/
>>>>>>>>>>>>
>>>>>>>>>>>> So i have to patch following 3 file manually?
>>>>>>>>>>>>
>>>>>>>>>>>> nova/tests/unit/virt/libvirt/test_driver.py213
>>>>>>>>>>>> nova/tests/unit/virt/test_virt_drivers.py2
>>>>>>>>>>>> nova/virt/libvirt/driver.py
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Aug 22, 2018 at 12:42 AM, Matthew Thode
>>>>>>>>>>>> <prometheanfire at gentoo.org> wrote:
>>>>>>>>>>>>> On 18-08-22 00:27:08, Satish Patel wrote:
>>>>>>>>>>>>>> Folks,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am running openstack queens and hypervisor is kvm, my live migration
>>>>>>>>>>>>>> working fine. but somehow it stuck to 8 Mb network speed and taking
>>>>>>>>>>>>>> long time to migrate 1G instance. I have 10Gbps network and i have
>>>>>>>>>>>>>> tried to copy 10G file between two compute node and it did copy in 2
>>>>>>>>>>>>>> minute, so i am not seeing any network issue also.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> it seem live_migration has some bandwidth limit, I have tried
>>>>>>>>>>>>>> following option in nova.conf but it didn't work
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> live_migration_bandwidth = 500
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My nova.conf look like following:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> live_migration_uri =
>>>>>>>>>>>>>> "qemu+ssh://nova@%s/system?no_verify=1&keyfile=/var/lib/nova/.ssh/id_rsa"
>>>>>>>>>>>>>> live_migration_tunnelled = True
>>>>>>>>>>>>>> live_migration_bandwidth = 500
>>>>>>>>>>>>>> hw_disk_discard = unmap
>>>>>>>>>>>>>> disk_cachemodes = network=writeback
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Do you have a this patch (and a couple of patches up to it)?
>>>>>>>>>>>>> https://bugs.launchpad.net/nova/+bug/1786346
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I don't know if that would cleanly apply (there are other patches that
>>>>>>>>>>> changed those functions within the last month and a half. It'd be best
>>>>>>>>>>> to upgrade and not do just one patch (which would be an untested
>>>>>>>>>>> process).
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The sha for nova has not been updated yet (next update is 24-48 hours
>>>>>>>>> away iirc), once that's done you can use the head of stable/queens from
>>>>>>>>> OSA and run a inter-series upgrade (but the minimal thing to do would be
>>>>>>>>> to run repo-build and os-nova plays). I'm not sure when that sha bump
>>>>>>>>> will be tagged in a full release if you would rather wait on that.
>>>>>>>
>>>>>>> it's this sha that needs updating.
>>>>>>> https://github.com/openstack/openstack-ansible/blob/stable/queens/playbooks/defaults/repo_packages/openstack_services.yml#L173
>>>>>>>
>>>>>
>>>>> I'm not sure how you are doing overrides, but set the following as an
>>>>> override, then rerun the repo-build playbook (to rebuild the nova venv)
>>>>> then rerun the nova playbook to install it.
>>>>>
>>>>> nova_git_install_branch: dee99b1ed03de4b6ded94f3cf6d2ea7214bca93b
>>>>>
>>>
>>> The sha I gave was head of the queens branch of openstack/nova. It's
>>> also the commit in that branch that containst the fix.
>>>
>
> I don't think that is known behavior, different memory pressure causing
> the difference maybe?
>
> --
> Matthew Thode (prometheanfire)
More information about the Openstack
mailing list