[Openstack-operators] Migrating instances in grizzly
Juan José Pavlik Salles
jjpavlik at gmail.com
Thu Sep 12 13:17:06 UTC 2013
I've also tried this patch https://bugs.launchpad.net/oslo/+bug/1189711 and
now i have a new message in nova-compute.log it might be related:
2013-09-12 13:02:39.116 4274 ERROR nova.servicegroup.drivers.db [-] model
server went away
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db Traceback
(most recent call last):
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py", line
92, in _report_state
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
service.service_ref, state_catalog)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 627, in
service_update
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db return
self.conductor_rpcapi.service_update(context, service, values)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 365, in
service_update
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db return
self.call(context, msg, version='1.34')
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py", line
80, in call
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db return
rpc.call(context, self._get_topic(topic), msg, timeout)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py",
line 140, in call
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db return
_get_impl().call(CONF, context, topic, msg, timeout)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py",
line 802, in call
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
"""Sends a message on a topic without waiting for a response."""
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
612, in call
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db rv =
list(rv)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
561, in __iter__
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db raise
result
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db TypeError:
exceptions must be old-style classes or derived from BaseException, not
NoneType
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db Traceback
(most recent call last):
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
430, in _process_data
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db rval =
self.proxy.dispatch(ctxt, version, method, **args)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py",
line 133, in dispatch
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db return
getattr(proxyobj, method)(ctxt, **kwargs)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py",
line 424, in inner
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db return
catch_client_exception(exceptions, func, *args, **kwargs)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py",
line 407, in catch_client_exception
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db return
func(*args, **kwargs)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 364, in
service_update
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db svc =
self.db.service_update(context, service['id'], values)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/db/api.py", line 169, in
service_update
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db return
IMPL.service_update(context, service_id, values)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 96, in
wrapper
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db return
f(*args, **kwargs)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 437, in
service_update
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
service_ref.save(session=session)
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db File
"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 452, in
__exit__
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db raise
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db TypeError:
exceptions must be old-style classes or derived from BaseException, not
NoneType
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:39.116 4274 TRACE nova.servicegroup.drivers.db
2013-09-12 13:02:49.175 4274 ERROR nova.servicegroup.drivers.db [-]
Recovered model server connection!
No clues, it could alse be becausa i miss applied the patch xD, who knows
haha.
2013/9/12 Juan José Pavlik Salles <jjpavlik at gmail.com>
> Hi Daneyon, I tried the patch you mention but didn't fix my problem. It
> seems to be something related to the network part of the live-migration,
> because the vm actually moves from one node to the other but the hole
> process never ends. Have anyone tried live-migration with a similar
> deployment?
>
>
> 2013/9/5 Juan José Pavlik Salles <jjpavlik at gmail.com>
>
>> I havent, ill take a look tomorrow, just in case. Ill run they same steps
>> and take a look at the log files.
>> El sep 5, 2013 12:50 p.m., "Daneyon Hansen (danehans)" <
>> danehans at cisco.com> escribió:
>>
>>
>>> Have you tried the patch for this bug?
>>>
>>> https://bugs.launchpad.net/oslo/+bug/856764
>>>
>>>
>>>
>>> Regards,
>>> Daneyon Hansen
>>> Software Engineer
>>> Email: danehans at cisco.com
>>> Phone: 303-718-0400
>>> http://about.me/daneyon_hansen
>>>
>>>
>>>
>>>
>>> On 9/5/13 7:02 AM, "Emilien Macchi" <emilien.macchi at enovance.com> wrote:
>>>
>>> >Hi,
>>> >
>>> >We have the same issue here with Grizzly 2013.1.2 / Ubuntu 12.04 /
>>> >libvirt 1.0.2.
>>> >
>>> >Which release are you running ?
>>> >
>>> >Emilien Macchi
>>> >----------------------------------------------------
>>> ># OpenStack Engineer
>>> >// eNovance Inc. http://enovance.com
>>> >// ✉ emilien at enovance.com ☎ +33 (0)1 49 70 99 80
>>> >// 10 rue de la Victoire 75009 Paris
>>> >
>>> >On Tue 03 Sep 2013 02:44:09 AM CEST, Juan José Pavlik Salles wrote:
>>> >> I've also found this in nova-conductor.log:
>>> >>
>>> >> 2013-09-02 15:35:27.208 DEBUG nova.openstack.common.rpc.common
>>> >> [req-e0473533-89af-4ff5-b6fa-4b0b6eb50a6d
>>> >> 31020076174943bdb7486c330a298d93 d1e3aae242f14c488d2225dc
>>> >> bf1e96d6] Timed out waiting for RPC response: timed out
>>> >> _error_callback
>>> >>
>>>
>>> >>/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py:
>>> >>628
>>> >> 2013-09-02 15:35:27.222 ERROR nova.openstack.common.rpc.amqp
>>> >> [req-e0473533-89af-4ff5-b6fa-4b0b6eb50a6d
>>> >> 31020076174943bdb7486c330a298d93 d1e3aae242f14c488d2225dcbf
>>> >> 1e96d6] Exception during message handling
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> Traceback (most recent call last):
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
>>> >> line 430, in _proce
>>> >> ss_data
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> rval = self.proxy.dispatch(ctxt, version, method, **args)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py
>>> >>",
>>> >> line 133, in
>>> >> dispatch
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> return getattr(proxyobj, method)(ctxt, **kwargs)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py",
>>> >> line 399, in network_migrat
>>> >> e_instance_start
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> self.network_api.migrate_instance_start(context, instance, migration)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 89,
>>> >> in wrapped
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> return func(self, context, *args, **kwargs)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 501,
>>> >> in migrate_instance_sta
>>> >> rt
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> self.network_rpcapi.migrate_instance_start(context, **args)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line
>>> >> 333, in migrate_instance_
>>> >> start
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> version='1.2')
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py",
>>> >> line 80, in call
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> return rpc.call(context, self._get_topic(topic), msg, timeout)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py",
>>> >> line 140, in ca
>>> >> ll
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> return _get_impl().call(CONF, context, topic, msg, timeout)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py
>>> >>",
>>> >> line 798, in
>>> >> call
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> rpc_amqp.get_connection_pool(conf, Connection))
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
>>> >> line 612, in call
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> rv = list(rv)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
>>> >> line 554, in __iter__
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> self.done()
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> self.gen.next()
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
>>> >> line 551, in __iter__
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> self._iterator.next()
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py
>>> >>",
>>> >> line 648, in iterconsume
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> yield self.ensure(_error_callback, _consume)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py
>>> >>",
>>> >> line 566, in ensure
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> error_callback(e)
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py
>>> >>",
>>> >> line 629, in _error_callback
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> raise rpc_common.Timeout()
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> Timeout: Timeout while waiting on RPC response.
>>> >> 2013-09-02 15:35:27.222 1363 TRACE nova.openstack.common.rpc.amqp
>>> >> 2013-09-02 15:35:27.237 ERROR nova.openstack.common.rpc.common
>>> >> [req-e0473533-89af-4ff5-b6fa-4b0b6eb50a6d
>>> >> 31020076174943bdb7486c330a298d93 d1e3aae242f14c488d2225dcbf1e96d6]
>>> >> Returning exception Timeout while waiting on RPC response. to caller
>>> >>
>>> >> Does anybody know all the steps that take to live-migrate an instance
>>> >> ?? It seems to be stopping inside the network_migrate_instance_start
>>> >> function, really no clue at all...
>>> >>
>>> >>
>>> >> 2013/9/2 Juan José Pavlik Salles <jjpavlik at gmail.com
>>> >> <mailto:jjpavlik at gmail.com>>
>>> >>
>>> >> Hi guys, last friday i started testing live-migration in my
>>> >> grizzly cloud with shared storage (gfs2) but i run into a problem,
>>> >> a little weird:
>>> >>
>>> >> This is the status before migrating:
>>> >>
>>> >> -I've p9 instances also called instance-00000022 running on
>>> >> "acelga" compute node.
>>> >>
>>> >> /root at acelga:~/tools# virsh list/
>>> >> / Id Name State/
>>> >> /----------------------------------------------------/
>>> >> / 6 instance-00000022 running/
>>> >> /
>>> >> /
>>> >> /root at acelga:~/tools# /
>>> >> /
>>> >> /
>>> >> /
>>> >> /
>>> >> /root at cebolla:~/tool# virsh list/
>>> >> / Id Nombre Estado/
>>> >> /----------------------------------------------------/
>>> >> /
>>> >> /
>>> >> /root at cebolla:~/tool# /
>>> >>
>>> >> -Here you can see all the info about the instance
>>> >>
>>> >> /root at cebolla:~/tool# nova --os-username=noc-admin
>>> >> --os-tenant-name=noc --os-password=XXXXXXX --os-auth-url
>>> >> http://172.19.136.1:35357/v2.0 show
>>> >> de2bcbed-f7b6-40cd-89ca-acf6fe2f2d09/
>>> >>
>>>
>>> >>/+-------------------------------------+---------------------------------
>>> >>--------------------------+/
>>> >> /| Property | Value
>>> >> |/
>>> >>
>>>
>>> >>/+-------------------------------------+---------------------------------
>>> >>--------------------------+/
>>> >> /| status | ACTIVE
>>> >> |/
>>> >> /| updated | 2013-09-02T15:27:39Z
>>> >> |/
>>> >> /| OS-EXT-STS:task_state | None
>>> >> |/
>>> >> /| OS-EXT-SRV-ATTR:host | acelga
>>> >> |/
>>> >> /| key_name | None
>>> >> |/
>>> >> /| image | Ubuntu 12.04.2 LTS
>>> >> (1359ca8d-23a2-40e8-940f-d90b3e68bb39) |/
>>> >> /| vlan1 network | 172.16.16.175
>>> >> |/
>>> >> /| hostId |
>>> >> 81be94870821e17e327d92e9c80548ffcdd37d24054a235116669f53 |/
>>> >> /| OS-EXT-STS:vm_state | active
>>> >> |/
>>> >> /| OS-EXT-SRV-ATTR:instance_name | instance-00000022
>>> >> |/
>>> >> /| OS-EXT-SRV-ATTR:hypervisor_hostname | acelga.psi.unc.edu.ar
>>> >> <http://acelga.psi.unc.edu.ar>
>>> >>|/
>>> >> /| flavor | m1.tiny (1)
>>> >> |/
>>> >> /| id |
>>> >> de2bcbed-f7b6-40cd-89ca-acf6fe2f2d09 |/
>>> >> /| security_groups | [{u'name': u'default'}]
>>> >> |/
>>> >> /| user_id |
>>> >> 20390b639d4449c18926dca5e038ec5e |/
>>> >> /| name | p9
>>> >> |/
>>> >> /| created | 2013-09-02T15:27:06Z
>>> >> |/
>>> >> /| tenant_id |
>>> >> d1e3aae242f14c488d2225dcbf1e96d6 |/
>>> >> /| OS-DCF:diskConfig | MANUAL
>>> >> |/
>>> >> /| metadata | {}
>>> >> |/
>>> >> /| accessIPv4 |
>>> >> |/
>>> >> /| accessIPv6 |
>>> >> |/
>>> >> /| progress | 0
>>> >> |/
>>> >> /| OS-EXT-STS:power_state | 1
>>> >> |/
>>> >> /| OS-EXT-AZ:availability_zone | nova
>>> >> |/
>>> >> /| config_drive |
>>> >> |/
>>> >>
>>>
>>> >>/+-------------------------------------+---------------------------------
>>> >>--------------------------+/
>>> >> /root at cebolla:~/tool#/
>>> >>
>>> >> -So i try to move it to the other node "cebolla"
>>> >>
>>> >> /root at acelga:~/tools# nova --os-username=noc-admin
>>> >> --os-tenant-name=noc --os-password=HjZ5V9yj --os-auth-url
>>> >> http://172.19.136.1:35357/v2.0 live-migration
>>> >> de2bcbed-f7b6-40cd-89ca-acf6fe2f2d09 cebolla
>>> >> /
>>> >> /root at acelga:~/tools# virsh list/
>>> >> / Id Name State/
>>> >> /----------------------------------------------------/
>>> >> /
>>> >> /
>>> >> /root at acelga:~/tools#/
>>> >>
>>> >> No error messages at all on "acelga" compute node so far. If i
>>> >> check the other node i can see the instance've been migrated
>>> >>
>>> >> /root at cebolla:~/tool# virsh list/
>>> >> / Id Nombre Estado/
>>> >> /----------------------------------------------------/
>>> >> / 11 instance-00000022 ejecutando/
>>> >> /
>>> >> /
>>> >> /root at cebolla:~/tool#/
>>> >>
>>> >>
>>> >> -BUT... after a few seconds i get this on "acelga"'s
>>> >>nova-compute.log
>>> >>
>>> >>
>>> >> /2013-09-02 15:35:45.784 4601 DEBUG
>>> >> nova.openstack.common.rpc.common [-] Timed out waiting for RPC
>>> >> response: timed out _error_callback
>>> >>
>>>
>>> >>/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py:
>>> >>628/
>>> >> /2013-09-02 15:35:45.790 4601 ERROR nova.utils [-] in fixed
>>> >> duration looping call/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils Traceback (most
>>> >> recent call last):/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >> "/usr/lib/python2.7/dist-packages/nova/utils.py", line 594, in
>>> >>_inner/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils
>>> >> self.f(*self.args, **self.kw <http://self.kw>)/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
>>> >> line 3129, in wait_for_live_migration/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils migrate_data)/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
>>> >> 3208, in _post_live_migration/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils migration)/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >> "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line
>>> >> 664, in network_migrate_instance_start/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils migration)/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >> "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line
>>> >> 415, in network_migrate_instance_start/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils return
>>> >> self.call(context, msg, version='1.41')/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py",
>>> >> line 80, in call/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils return
>>> >> rpc.call(context, self._get_topic(topic), msg, timeout)/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py",
>>> >> line 140, in call/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils return
>>> >> _get_impl().call(CONF, context, topic, msg, timeout)/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py
>>> >>",
>>> >> line 798, in call/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils
>>> >> rpc_amqp.get_connection_pool(conf, Connection))/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
>>> >> line 612, in call/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils rv = list(rv)/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
>>> >> line 554, in __iter__/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils self.done()/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils
>>> self.gen.next()/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
>>> >> line 551, in __iter__/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils
>>> >> self._iterator.next()/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py
>>> >>",
>>> >> line 648, in iterconsume/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils yield
>>> >> self.ensure(_error_callback, _consume)/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py
>>> >>",
>>> >> line 566, in ensure/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils
>>> >>error_callback(e)/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils File
>>> >>
>>>
>>> >>"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py
>>> >>",
>>> >> line 629, in _error_callback/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils raise
>>> >> rpc_common.Timeout()/
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils Timeout: Timeout
>>> >> while waiting on RPC response./
>>> >> /2013-09-02 15:35:45.790 4601 TRACE nova.utils/
>>> >>
>>> >>
>>> >> -And the VM state never changes back to ACTIVE from MIGRATING:
>>> >>
>>> >>
>>> >> /root at cebolla:~/tool# nova --os-username=noc-admin
>>> >> --os-tenant-name=noc --os-password=XXXXX --os-auth-url
>>> >> http://172.19.136.1:35357/v2.0 show
>>> >> de2bcbed-f7b6-40cd-89ca-acf6fe2f2d09/
>>> >>
>>>
>>> >>/+-------------------------------------+---------------------------------
>>> >>--------------------------+/
>>> >> /| Property | Value
>>> >> |/
>>> >>
>>>
>>> >>/+-------------------------------------+---------------------------------
>>> >>--------------------------+/
>>> >> /| status | MIGRATING
>>> >> |/
>>> >> /| updated | 2013-09-02T15:33:54Z
>>> >> |/
>>> >> /| OS-EXT-STS:task_state | migrating
>>> >> |/
>>> >> /| OS-EXT-SRV-ATTR:host | acelga
>>> >> |/
>>> >> /| key_name | None
>>> >> |/
>>> >> /| image | Ubuntu 12.04.2 LTS
>>> >> (1359ca8d-23a2-40e8-940f-d90b3e68bb39) |/
>>> >> /| vlan1 network | 172.16.16.175
>>> >> |/
>>> >> /| hostId |
>>> >> 81be94870821e17e327d92e9c80548ffcdd37d24054a235116669f53 |/
>>> >> /| OS-EXT-STS:vm_state | active
>>> >> |/
>>> >> /| OS-EXT-SRV-ATTR:instance_name | instance-00000022
>>> >> |/
>>> >> /| OS-EXT-SRV-ATTR:hypervisor_hostname | acelga.psi.unc.edu.ar
>>> >> <http://acelga.psi.unc.edu.ar>
>>> >>|/
>>> >> /| flavor | m1.tiny (1)
>>> >> |/
>>> >> /| id |
>>> >> de2bcbed-f7b6-40cd-89ca-acf6fe2f2d09 |/
>>> >> /| security_groups | [{u'name': u'default'}]
>>> >> |/
>>> >> /| user_id |
>>> >> 20390b639d4449c18926dca5e038ec5e |/
>>> >> /| name | p9
>>> >> |/
>>> >> /| created | 2013-09-02T15:27:06Z
>>> >> |/
>>> >> /| tenant_id |
>>> >> d1e3aae242f14c488d2225dcbf1e96d6 |/
>>> >> /| OS-DCF:diskConfig | MANUAL
>>> >> |/
>>> >> /| metadata | {}
>>> >> |/
>>> >> /| accessIPv4 |
>>> >> |/
>>> >> /| accessIPv6 |
>>> >> |/
>>> >> /| OS-EXT-STS:power_state | 1
>>> >> |/
>>> >> /| OS-EXT-AZ:availability_zone | nova
>>> >> |/
>>> >> /| config_drive |
>>> >> |/
>>> >>
>>>
>>> >>/+-------------------------------------+---------------------------------
>>> >>--------------------------+/
>>> >> /root at cebolla:~/tool#/
>>> >>
>>> >>
>>> >> Funny fact:
>>> >> -The vm still answer ping after migration, so i think this is
>>> good.
>>> >>
>>> >> Any ideas about this problem? At first i thought it could be
>>> >> related to a connection problem between the nodes, but the VM
>>> >> migrates completly in hipervisor level somehow there is some
>>> >> "instance've been migrated ACK" missing.
>>> >>
>>> >>
>>> >> --
>>> >> Pavlik Salles Juan José
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Pavlik Salles Juan José
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> OpenStack-operators mailing list
>>> >> OpenStack-operators at lists.openstack.org
>>> >>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>> >
>>>
>>>
>
>
> --
> Pavlik Salles Juan José
>
--
Pavlik Salles Juan José
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20130912/63bc3cb5/attachment.html>
More information about the OpenStack-operators
mailing list