Open Stack

I have some Nova problems in a multi-host Devstack setup (from Grizzly trunk, one controller node – named Controller, two compute hosts – named ComputeNode1 and ComputeNode2) and I'm confused. Spinning up VMs fails somewhat randomly and when failure happens I see error messages as the one below. It was taken from host ComputeNode2 and what strikes me in that error message chain is that something for ComputeNode1 seems to have ended up in ComputeNode2. Something seems wrong with the RPC.

Any ideas?


012-12-13 17:44:11 ERROR nova.openstack.common.rpc.amqp [req-eec1a340-d740-46d3-906d-4654c43bd798 admin demo] Exception during message handling
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 277, in _process_data
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     rval = self.proxy.dispatch(ctxt, version, method, **args)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 147, in dispatch
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     return getattr(proxyobj, method)(ctxt, **kwargs)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/exception.py", line 115, in wrapped
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     temp_level, payload)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     self.gen.next()
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/exception.py", line 90, in wrapped
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     return f(*args, **kw)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 212, in decorated_function
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     pass
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     self.gen.next()
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 198, in decorated_function
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 239, in decorated_function
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     kwargs['instance']['uuid'], e, sys.exc_info())
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     self.gen.next()
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 227, in decorated_function
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 973, in run_instance
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     do_run_instance()
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 228, in inner
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     retval = f(*args, **kwargs)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 972, in do_run_instance
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     admin_password, is_first_time, node, instance)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 646, in _run_instance
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     self._set_instance_error_state(context, instance['uuid'])
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     self.gen.next()
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 609, in _run_instance
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     rt = self._get_resource_tracker(node)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 333, in _get_resource_tracker
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp     raise exception.NovaException(msg)
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp NovaException: ComputeNode1 is not a valid node managed by this compute host.
2012-12-13 17:44:11 TRACE nova.openstack.common.rpc.amqp
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121213/6be7328a/attachment.html>