Hi:<div><br></div><div>Have you ensured your messaging queue is working correctly?</div><div><br></div><div>Regards,<br clear="all"><div>JuanFra</div>
<br><br><div class="gmail_quote">2013/2/25 Javier Alvarez <span dir="ltr"><<a href="mailto:javier.alvarez@bsc.es" target="_blank">javier.alvarez@bsc.es</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello,<br>
<br>
I'm trying to setup OpenStack Essex on Debian and I'm getting an error when creating new VMs. The problem arises when trying to allocate the network of the VM, but I'm not sure why it's happening, I'm using nova-network, and that's the log output:<br>
<br>
2013-02-25 18:20:28 AUDIT nova.compute.manager [req-4d211944-bd5c-4e88-a8df-<u></u>c4a9167676e2 36a7e60b4d134307b61e06949d3373<u></u>5e 9558f53959a04cc992c3f8b6d91bfb<u></u>9f] [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] Starting instance...<br>
2013-02-25 18:20:40 INFO nova.virt.libvirt.connection [-] Compute_service record updated for bscgrid21<br>
2013-02-25 18:20:40 INFO nova.compute.manager [-] Updating host status<br>
2013-02-25 18:21:28 ERROR nova.rpc.common [req-4d211944-bd5c-4e88-a8df-<u></u>c4a9167676e2 36a7e60b4d134307b61e06949d3373<u></u>5e 9558f53959a04cc992c3f8b6d91bfb<u></u>9f] Timed out waiting for RPC response: timed out<br>
2013-02-25 18:21:28 TRACE nova.rpc.common Traceback (most recent call last):<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 490, in ensure<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return method(*args, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 567, in _consume<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return self.connection.drain_events(<u></u>timeout=timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File "/usr/lib/python2.7/dist-<u></u>packages/kombu/connection.py", line 167, in drain_events<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return self.transport.drain_events(<u></u>self.connection, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File "/usr/lib/python2.7/dist-<u></u>packages/kombu/transport/<u></u>amqplib.py", line 262, in drain_events<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return connection.drain_events(**<u></u>kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File "/usr/lib/python2.7/dist-<u></u>packages/kombu/transport/<u></u>amqplib.py", line 94, in drain_events<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return self.wait_multi(self.channels.<u></u>values(), timeout=timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File "/usr/lib/python2.7/dist-<u></u>packages/kombu/transport/<u></u>amqplib.py", line 100, in wait_multi<br>
2013-02-25 18:21:28 TRACE nova.rpc.common chanmap.keys(), allowed_methods, timeout=timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File "/usr/lib/python2.7/dist-<u></u>packages/kombu/transport/<u></u>amqplib.py", line 159, in _wait_multiple<br>
2013-02-25 18:21:28 TRACE nova.rpc.common channel, method_sig, args, content = read_timeout(timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File "/usr/lib/python2.7/dist-<u></u>packages/kombu/transport/<u></u>amqplib.py", line 132, in read_timeout<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return self.method_reader.read_<u></u>method()<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File "/usr/lib/python2.7/dist-<u></u>packages/amqplib/client_0_8/<u></u>method_framing.py", line 221, in read_method<br>
2013-02-25 18:21:28 TRACE nova.rpc.common raise m<br>
2013-02-25 18:21:28 TRACE nova.rpc.common timeout: timed out<br>
2013-02-25 18:21:28 TRACE nova.rpc.common<br>
2013-02-25 18:21:28 ERROR nova.compute.manager [req-4d211944-bd5c-4e88-a8df-<u></u>c4a9167676e2 36a7e60b4d134307b61e06949d3373<u></u>5e 9558f53959a04cc992c3f8b6d91bfb<u></u>9f] [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] Instance failed network setup<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] Traceback (most recent call last):<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/compute/manager.<u></u>py", line 571, in _allocate_network<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] requested_networks=requested_<u></u>networks)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/network/api.py", line 178, in allocate_for_instance<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] 'args': args})<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/__init__.py"<u></u>, line 68, in call<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] return _get_impl().call(context, topic, msg, timeout)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 674, in call<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] return rpc_amqp.call(context, topic, msg, timeout, Connection.pool)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/amqp.py", line 343, in call<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] rv = list(rv)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/amqp.py", line 304, in __iter__<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] self.done()<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/<u></u>contextlib.py", line 24, in __exit__<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] self.gen.next()<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/amqp.py", line 301, in __iter__<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] self._iterator.next()<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 572, in iterconsume<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] yield self.ensure(_error_callback, _consume)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 503, in ensure<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] error_callback(e)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 553, in _error_callback<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] raise rpc_common.Timeout()<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60] Timeout: Timeout while waiting on RPC response.<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance: 8c5b2d4a-c715-4f4d-8d74-<u></u>a6bdccbe1b60]<br>
2013-02-25 18:21:28 ERROR nova.rpc.amqp [req-4d211944-bd5c-4e88-a8df-<u></u>c4a9167676e2 36a7e60b4d134307b61e06949d3373<u></u>5e 9558f53959a04cc992c3f8b6d91bfb<u></u>9f] Exception during message handling<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp Traceback (most recent call last):<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/amqp.py", line 253, in _process_data<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp rval = node_func(context=ctxt, **node_args)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/exception.py", line 114, in wrapped<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp return f(*args, **kw)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/compute/manager.<u></u>py", line 183, in decorated_function<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp sys.exc_info())<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/<u></u>contextlib.py", line 24, in __exit__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self.gen.next()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/compute/manager.<u></u>py", line 177, in decorated_function<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp return function(self, context, instance_uuid, *args, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/compute/manager.<u></u>py", line 663, in run_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp do_run_instance()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/utils.py", line 945, in inner<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp retval = f(*args, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/compute/manager.<u></u>py", line 662, in do_run_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self._run_instance(context, instance_uuid, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/compute/manager.<u></u>py", line 463, in _run_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self._set_instance_error_<u></u>state(context, instance_uuid)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/<u></u>contextlib.py", line 24, in __exit__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self.gen.next()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/compute/manager.<u></u>py", line 436, in _run_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp requested_networks)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/compute/manager.<u></u>py", line 571, in _allocate_network<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp requested_networks=requested_<u></u>networks)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/network/api.py", line 178, in allocate_for_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp 'args': args})<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/__init__.py"<u></u>, line 68, in call<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp return _get_impl().call(context, topic, msg, timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 674, in call<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp return rpc_amqp.call(context, topic, msg, timeout, Connection.pool)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/amqp.py", line 343, in call<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp rv = list(rv)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/amqp.py", line 304, in __iter__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self.done()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/<u></u>contextlib.py", line 24, in __exit__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self.gen.next()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/amqp.py", line 301, in __iter__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self._iterator.next()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 572, in iterconsume<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp yield self.ensure(_error_callback, _consume)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 503, in ensure<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp error_callback(e)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-<u></u>packages/nova/rpc/impl_kombu.<u></u>py", line 553, in _error_callback<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp raise rpc_common.Timeout()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp Timeout: Timeout while waiting on RPC response.<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp<br>
<br>
Thanks in advance,<br>
<br>
Javi<br>
<br>
-- <br>
Javier Álvarez Cid-Fuentes<br>
Grid Computing and Clusters Group<br>
Barcelona Supercomputing Center (BSC-CNS)<br>
Tel. (+34) 93 413 72 46<br>
<br>
<br>
WARNING / LEGAL TEXT: This message is intended only for the use of the<br>
individual or entity to which it is addressed and may contain<br>
information which is privileged, confidential, proprietary, or exempt<br>
from disclosure under applicable law. If you are not the intended<br>
recipient or the person responsible for delivering the message to the<br>
intended recipient, you are strictly prohibited from disclosing,<br>
distributing, copying, or in any way using this message. If you have<br>
received this communication in error, please notify the sender and<br>
destroy and delete any copies you may have received.<br>
<br>
<a href="http://www.bsc.es/disclaimer" target="_blank">http://www.bsc.es/disclaimer</a><br>
<br>
______________________________<u></u>_________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~<u></u>openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~<u></u>openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/<u></u>ListHelp</a><br>
</blockquote></div><br></div>