[Openstack] Live Migration of VMs without shared storage
somshekar kadam
som_kadam at yahoo.co.in
Mon Mar 2 16:12:24 UTC 2015
Hello John,
Thanks for all your valuable inputs.
I had missed these configuration. Hope these configs mentioned below will be useful for others who are trying out live migration.
1. force_config_drive = none , otherwise it was giving error.
2. allow_migrate_to_same_host = False
allow_resize_to_same_host = False
3. set proper cpu mode as below
cpu_mode = custom
cpu_model = Nehalem
virt_type = kvm
Now I am able to make normal migration working fine, from one host to another.
When I try with live migration with block-migration, its failing for not getting proper host key verification ,,
I am looking into it. I have shared keys proper I am able to ssh to-fro host nodes using passwordless ssh.
no clue right now.
error log
------------------[[01;31mLive Migration fa ilure: operation failed: Failed to connect to remote libvirt URI qemu+ssh://stack@celestial8/system: Cannot recv data: Host key verifica tion failed.: Connection reset by peer^[[00m
on thread notification^[[00m ^[[00;33mfrom (pid=8953) thread_finished /opt/stack/nova/nova/virt/libvirt/driver.py:5638^[[00m
270370 Traceback (most recent call last):
270371 File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457, in fire_timers
270372 timer()
270373 File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
270374 cb(*args, **kw)
270375 File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 168, in _do_send
270376 waiter.switch(result)
270377 File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
270378 result = function(*args, **kwargs)
270379 File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5435, in _live_migration_operation
270380 instance=instance)
270381 File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 82, in __exit__
270382 six.reraise(self.type_, self.value, self.tb)
270383 File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5404, in _live_migration_operation
270384 CONF.libvirt.live_migration_bandwidth)
270385 File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
270386 result = proxy_call(self._autowrap, f, *args, **kwargs)
270387 File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
270388 rv = execute(f, *args, **kwargs)
270389 File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
270390 six.reraise(c, e, tb)
270391 File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
270392 rv = meth(*args, **kwargs)
270393 File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1582, in migrateToURI2
270394 if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self)
270395 libvirtError: operation failed: Failed to connect to remote libvirt URI qemu+ssh://stack@celestial8/system: Cannot recv data: Host key v erification failed.: Connection reset by peer
@
-----
Regards
Neelu
On Friday, 27 February 2015 8:09 PM, somshekar kadam <som_kadam at yahoo.co.in> wrote:
Have done the same configs as mentioned in above thread links for libvirt and nova.conf.
Regards
Neelu
On Friday, 27 February 2015 8:07 PM, somshekar kadam <som_kadam at yahoo.co.in> wrote:
John, your patch is in the mainline now. I have tested it. Not sure what is the thing I am missing
I have specified cpu model none and even tried setting cpu mode custom also and model kvm64.
I am getting below error on live migration.
error log
----
Remote error: libvirtError Requested operation is not valid: no CPU model specified [u’Traceback (most recent call last):\n’, u’ File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py”, line 142, in _dispatch_and_reply\n executo
Code
500
Details
File “/opt/stack/nova/nova/conductor/manager.py”, line 606, in _live_migrate block_migration, disk_over_commit) File “/opt/stack/nova/nova/conductor/tasks/live_migrate.py”, line 194, in execute return task.execute() File “/opt/stack/nova/nova/conductor/tasks/live_migrate.py”, line 62, in execute self._check_requested_destination() File “/opt/stack/nova/nova/conductor/tasks/live_migrate.py”, line 100, in _check_requested_destination self._call_livem_checks_on_host(self.destination) File “/opt/stack/nova/nova/conductor/tasks/live_migrate.py”, line 142, in _call_livem_checks_on_host destination, self.block_migration, self.disk_over_commit) File “/opt/stack/nova/nova/compute/rpcapi.py”, line 391, in check_can_live_migrate_destination disk_over_commit=disk_over_commit) File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py”, line 156, in call retry=self.retry) File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py”, line 90, in _send timeout=timeout, retry=retry) File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py”, line 417, in send retry=retry) File “/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py”, line 408, in _send raise result
Created
--------------------
debug log
---------------
DEBUG (shell:803) Live migration of instance 477e8963-aadf-4dc7-b26f- f6e9332dd33b to host celestial5 failed (HTTP 500) (Request-ID: req-7973edbc-8844-41c5-926a- 9f5e7cee62f2)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/ dist-packages/novaclient/ shell.py", line 800, in main
OpenStackComputeShell().main( argv)
File "/usr/local/lib/python2.7/ dist-packages/novaclient/ shell.py", line 730, in main
args.func(self.cs, args)
File "/usr/local/lib/python2.7/ dist-packages/novaclient/v1_1/ shell.py", line 3001, in do_live_migration
args.disk_over_commit)
File "/usr/local/lib/python2.7/ dist-packages/novaclient/v1_1/ servers.py", line 344, in live_migrate
disk_over_commit)
File "/usr/local/lib/python2.7/ dist-packages/novaclient/v1_1/ servers.py", line 1124, in live_migrate
'disk_over_commit': disk_over_commit})
File "/usr/local/lib/python2.7/ dist-packages/novaclient/v1_1/ servers.py", line 1240, in _action
return self.api.client.post(url, body=body)
File "/usr/local/lib/python2.7/ dist-packages/novaclient/ client.py", line 490, in post
return self._cs_request(url, 'POST', **kwargs)
File "/usr/local/lib/python2.7/ dist-packages/novaclient/ client.py", line 465, in _cs_request
resp, body = self._time_request(url, method, **kwargs)
File "/usr/local/lib/python2.7/ dist-packages/novaclient/ client.py", line 439, in _time_request
resp, body = self.request(url, method, **kwargs)
File "/usr/local/lib/python2.7/ dist-packages/novaclient/ client.py", line 433, in request
raise exceptions.from_response(resp, body, url, method)
ClientException: Live migration of instance 477e8963-aadf-4dc7-b26f- f6e9332dd33b to host celestial5 failed (HTTP 500) (Request-ID: req-7973edbc-8844-41c5-926a- 9f5e7cee62f2)
ERROR (ClientException): Live migration of instance 477e8963-aadf-4dc7-b26f- f6e9332dd33b to host celestial5 failed (HTTP 500) (Request-ID: req-7973edbc-8844-41c5-926a- 9f5e7cee62f2)
----------------------
Regards
Neelu
On Thursday, 26 February 2015 8:21 PM, John Griffith <john.griffith at solidfire.com> wrote:
On Thu, Feb 26, 2015 at 7:07 AM, somshekar kadam <som_kadam at yahoo.co.in> wrote:
First of all thanks for quick reply.
Documentation does not mention much about Volume-backed live migration.
Is it tested, I mean supported and working.
I will try Block live migration as no storage is required.
Regards
Neelu
On Thursday, 26 February 2015 7:13 PM, Robert van Leeuwen <Robert.vanLeeuwen at spilgames.com> wrote:
> Is the Live Migration of VMs even without shared storage supported in Openstack now.
> If yes is there any document for the same.
Wel, depends on what you call "supported".
Yes, it is possible.
Will it always work? Probably not until you look at the bugs below.
They have been fixed recently but they might not be merged with the version you are running:
https://bugs.launchpad.net/nova/+bug/1270825
https://bugs.launchpad.net/nova/+bug/1082414
There might be more issues but I hit the ones mentioned above.
Have a look at the docs to configure it:
http://docs.openstack.org/admin-guide-cloud/content/section_configuring-compute-migrations.html
Cheers,
Robert van Leeuwen
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack at lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
It's dated and getting the patches backported has proven to take forever, but I did a write up and testing on this a while back [1]. Should still be accurate.
Thanks,John
[1]: https://griffithscorner.wordpress.com/2014/12/08/openstack-live-migration-with-cinder-backed-instances/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150302/abbbec86/attachment.html>
More information about the Openstack
mailing list