[Openstack] Live Migration of VMs without shared storage

somshekar kadam som_kadam at yahoo.co.in
Mon Mar 2 16:12:24 UTC 2015


Hello John, 
Thanks for all your valuable inputs. 
I had missed these configuration. Hope these configs mentioned below will be useful for others who are trying out live migration. 

1. force_config_drive = none , otherwise it was giving error. 
2. allow_migrate_to_same_host = False
allow_resize_to_same_host = False

3. set proper cpu mode as below 
cpu_mode = custom
cpu_model = Nehalem
virt_type = kvm

Now I am able to make normal migration working fine, from one host to another. 

When I try with live migration with block-migration, its failing for not getting proper host key verification ,, 
I am looking into it. I have shared keys proper I am able to ssh to-fro host nodes using passwordless ssh. 
no clue right now. 

error log 
------------------[[01;31mLive Migration fa       ilure: operation failed: Failed to connect to remote libvirt URI qemu+ssh://stack@celestial8/system: Cannot recv data: Host key verifica       tion failed.: Connection reset by peer^[[00m


      on thread notification^[[00m ^[[00;33mfrom (pid=8953) thread_finished /opt/stack/nova/nova/virt/libvirt/driver.py:5638^[[00m
270370 Traceback (most recent call last):
270371   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457, in fire_timers
270372     timer()
270373   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
270374     cb(*args, **kw)
270375   File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 168, in _do_send
270376     waiter.switch(result)
270377   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
270378     result = function(*args, **kwargs)
270379   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5435, in _live_migration_operation
270380     instance=instance)
270381   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 82, in __exit__
270382     six.reraise(self.type_, self.value, self.tb)
270383   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5404, in _live_migration_operation
270384     CONF.libvirt.live_migration_bandwidth)
270385   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
270386     result = proxy_call(self._autowrap, f, *args, **kwargs)
270387   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
270388     rv = execute(f, *args, **kwargs)
270389   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
270390     six.reraise(c, e, tb)
270391   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
270392     rv = meth(*args, **kwargs)
270393   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1582, in migrateToURI2
270394     if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self)
270395 libvirtError: operation failed: Failed to connect to remote libvirt URI qemu+ssh://stack@celestial8/system: Cannot recv data: Host key v       erification failed.: Connection reset by peer
@                                                                 
-----

 
Regards
Neelu 

     On Friday, 27 February 2015 8:09 PM, somshekar kadam <som_kadam at yahoo.co.in> wrote:
   

 Have done the same configs as mentioned in above thread links for libvirt and nova.conf. 
 
Regards
Neelu 

     On Friday, 27 February 2015 8:07 PM, somshekar kadam <som_kadam at yahoo.co.in> wrote:
   

 John, your patch is in the mainline now. I have tested it. Not sure what is the thing I am missing
I have specified cpu model none and even tried setting cpu mode custom also and model kvm64. 

I am getting below error on live migration. 

error log
----
Remote error: libvirtError Requested operation is not valid: no CPU model specified [u’Traceback (most recent call last):\n’, u’ File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py”, line 142, in _dispatch_and_reply\n executo
Code
500
Details
File “/opt/stack/nova/nova/conductor/manager.py”, line 606, in _live_migrate block_migration, disk_over_commit) File “/opt/stack/nova/nova/conductor/tasks/live_migrate.py”, line 194, in execute return task.execute() File “/opt/stack/nova/nova/conductor/tasks/live_migrate.py”, line 62, in execute self._check_requested_destination() File “/opt/stack/nova/nova/conductor/tasks/live_migrate.py”, line 100, in _check_requested_destination self._call_livem_checks_on_host(self.destination) File “/opt/stack/nova/nova/conductor/tasks/live_migrate.py”, line 142, in _call_livem_checks_on_host destination, self.block_migration, self.disk_over_commit) File “/opt/stack/nova/nova/compute/rpcapi.py”, line 391, in check_can_live_migrate_destination disk_over_commit=disk_over_commit) File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py”, line 156, in call retry=self.retry) File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py”, line 90, in _send timeout=timeout, retry=retry) File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py”, line 417, in send retry=retry) File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py”, line 408, in _send raise result
Created
--------------------

debug log 
---------------
DEBUG (shell:803) Live migration of instance 477e8963-aadf-4dc7-b26f- f6e9332dd33b to host celestial5 failed (HTTP 500) (Request-ID: req-7973edbc-8844-41c5-926a- 9f5e7cee62f2)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/ shell.py", line 800, in main
    OpenStackComputeShell().main( argv)
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/ shell.py", line 730, in main
    args.func(self.cs, args)
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/v1_1/ shell.py", line 3001, in do_live_migration
    args.disk_over_commit)
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/v1_1/ servers.py", line 344, in live_migrate
    disk_over_commit)
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/v1_1/ servers.py", line 1124, in live_migrate
    'disk_over_commit': disk_over_commit})
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/v1_1/ servers.py", line 1240, in _action
    return self.api.client.post(url, body=body)
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/ client.py", line 490, in post
    return self._cs_request(url, 'POST', **kwargs)
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/ client.py", line 465, in _cs_request
    resp, body = self._time_request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/ client.py", line 439, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/ dist-packages/novaclient/ client.py", line 433, in request
    raise exceptions.from_response(resp, body, url, method)
ClientException: Live migration of instance 477e8963-aadf-4dc7-b26f- f6e9332dd33b to host celestial5 failed (HTTP 500) (Request-ID: req-7973edbc-8844-41c5-926a- 9f5e7cee62f2)
ERROR (ClientException): Live migration of instance 477e8963-aadf-4dc7-b26f- f6e9332dd33b to host celestial5 failed (HTTP 500) (Request-ID: req-7973edbc-8844-41c5-926a- 9f5e7cee62f2)
----------------------

Regards
Neelu 

     On Thursday, 26 February 2015 8:21 PM, John Griffith <john.griffith at solidfire.com> wrote:
   

 

On Thu, Feb 26, 2015 at 7:07 AM, somshekar kadam <som_kadam at yahoo.co.in> wrote:

First of all thanks for quick reply. 
Documentation does not mention much about Volume-backed live migration. 
Is it tested, I mean supported and working. 
I will try Block live migration as no storage is required. 
 
Regards
Neelu 

     On Thursday, 26 February 2015 7:13 PM, Robert van Leeuwen <Robert.vanLeeuwen at spilgames.com> wrote:
   

 > Is the Live Migration of VMs even without shared storage supported in Openstack now.
> If yes is there any document for the same.

Wel, depends on what you call "supported".
Yes, it is possible. 
Will it always work? Probably not until you look at the bugs below.
They have been fixed recently but they might not be merged with the version you are running: 
https://bugs.launchpad.net/nova/+bug/1270825
https://bugs.launchpad.net/nova/+bug/1082414

There might be more issues but I hit the ones mentioned above.
Have a look at the docs to configure it:
http://docs.openstack.org/admin-guide-cloud/content/section_configuring-compute-migrations.html

Cheers,
Robert van Leeuwen



 












    
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



It's dated and getting the patches backported has proven to take forever, but I did a write up and testing on this a while back [1].  Should still be accurate.
Thanks,John
[1]: https://griffithscorner.wordpress.com/2014/12/08/openstack-live-migration-with-cinder-backed-instances/

    

    

   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150302/abbbec86/attachment.html>


More information about the Openstack mailing list