Javier,<div><br></div><div>The stack trace you provided indicates that the allocate network call from Compute (to nova-network) timed out. I would suggest checking the nova-network logs, that might give some insight.</div>
<div>Also, IMO, the create instance request is being served by Compute means the RabbitMQ is working. You might want to ensure that nova-network has also connected to the same AMPQ server (as Compute is).</div><div><br></div>
<div>Regards,</div><div>Unmesh G.<br><br><div class="gmail_quote">On Tue, Feb 26, 2013 at 2:16 PM, Javier Alvarez <span dir="ltr"><<a href="mailto:javier.alvarez@bsc.es" target="_blank">javier.alvarez@bsc.es</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hello JuanFra,<br>
<br>
How can I check that? I have rabbitmq-server running on the
controller node and I have set rabbit_host property in the .conf
files. Also, if I restart nova-compute service, I get the
following message:<br>
<br>
2013-02-26 09:43:52 INFO nova.rpc.common [-] Connected to AMQP
server on <a href="http://172.16.8.1:5672" target="_blank">172.16.8.1:5672</a><br>
<br>
Is there any other test I could do to verify everything is
properly working?<br>
<br>
Thanks,<br>
<br>
Javi<div><div class="h5"><br>
<br>
On 25/02/13 20:09, JuanFra Rodriguez Cardoso wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">Hi:
<div><br>
</div>
<div>Have you ensured your messaging queue is working correctly?</div>
<div><br>
</div>
<div>Regards,<br clear="all">
<div>JuanFra</div>
<br>
<br>
<div class="gmail_quote">2013/2/25 Javier Alvarez <span dir="ltr"><<a href="mailto:javier.alvarez@bsc.es" target="_blank">javier.alvarez@bsc.es</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello,<br>
<br>
I'm trying to setup OpenStack Essex on Debian and I'm
getting an error when creating new VMs. The problem arises
when trying to allocate the network of the VM, but I'm not
sure why it's happening, I'm using nova-network, and that's
the log output:<br>
<br>
2013-02-25 18:20:28 AUDIT nova.compute.manager
[req-4d211944-bd5c-4e88-a8df-c4a9167676e2
36a7e60b4d134307b61e06949d33735e
9558f53959a04cc992c3f8b6d91bfb9f] [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Starting instance...<br>
2013-02-25 18:20:40 INFO nova.virt.libvirt.connection [-]
Compute_service record updated for bscgrid21<br>
2013-02-25 18:20:40 INFO nova.compute.manager [-] Updating
host status<br>
2013-02-25 18:21:28 ERROR nova.rpc.common
[req-4d211944-bd5c-4e88-a8df-c4a9167676e2
36a7e60b4d134307b61e06949d33735e
9558f53959a04cc992c3f8b6d91bfb9f] Timed out waiting for RPC
response: timed out<br>
2013-02-25 18:21:28 TRACE nova.rpc.common Traceback (most
recent call last):<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 490, in ensure<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return
method(*args, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 567, in _consume<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return
self.connection.drain_events(timeout=timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/connection.py", line
167, in drain_events<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return
self.transport.drain_events(self.connection, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
line 262, in drain_events<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return
connection.drain_events(**kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
line 94, in drain_events<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return
self.wait_multi(self.channels.values(), timeout=timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
line 100, in wait_multi<br>
2013-02-25 18:21:28 TRACE nova.rpc.common
chanmap.keys(), allowed_methods, timeout=timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
line 159, in _wait_multiple<br>
2013-02-25 18:21:28 TRACE nova.rpc.common channel,
method_sig, args, content = read_timeout(timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
line 132, in read_timeout<br>
2013-02-25 18:21:28 TRACE nova.rpc.common return
self.method_reader.read_method()<br>
2013-02-25 18:21:28 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/method_framing.py",
line 221, in read_method<br>
2013-02-25 18:21:28 TRACE nova.rpc.common raise m<br>
2013-02-25 18:21:28 TRACE nova.rpc.common timeout: timed out<br>
2013-02-25 18:21:28 TRACE nova.rpc.common<br>
2013-02-25 18:21:28 ERROR nova.compute.manager
[req-4d211944-bd5c-4e88-a8df-c4a9167676e2
36a7e60b4d134307b61e06949d33735e
9558f53959a04cc992c3f8b6d91bfb9f] [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Instance failed
network setup<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Traceback (most recent
call last):<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 571, in _allocate_network<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]
requested_networks=requested_networks)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/network/api.py", line
178, in allocate_for_instance<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] 'args': args})<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/rpc/__init__.py",
line 68, in call<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] return
_get_impl().call(context, topic, msg, timeout)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 674, in call<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] return
rpc_amqp.call(context, topic, msg, timeout, Connection.pool)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line
343, in call<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] rv = list(rv)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line
304, in __iter__<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] self.done()<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] self.gen.next()<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line
301, in __iter__<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]
self._iterator.next()<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 572, in iterconsume<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] yield
self.ensure(_error_callback, _consume)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 503, in ensure<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] error_callback(e)<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 553, in _error_callback<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] raise
rpc_common.Timeout()<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Timeout: Timeout while
waiting on RPC response.<br>
2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]<br>
2013-02-25 18:21:28 ERROR nova.rpc.amqp
[req-4d211944-bd5c-4e88-a8df-c4a9167676e2
36a7e60b4d134307b61e06949d33735e
9558f53959a04cc992c3f8b6d91bfb9f] Exception during message
handling<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp Traceback (most
recent call last):<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line
253, in _process_data<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp rval =
node_func(context=ctxt, **node_args)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line
114, in wrapped<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp return f(*args,
**kw)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 183, in decorated_function<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp sys.exc_info())<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self.gen.next()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 177, in decorated_function<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp return
function(self, context, instance_uuid, *args, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 663, in run_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp
do_run_instance()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 945,
in inner<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp retval =
f(*args, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 662, in do_run_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp
self._run_instance(context, instance_uuid, **kwargs)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 463, in _run_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp
self._set_instance_error_state(context, instance_uuid)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self.gen.next()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 436, in _run_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp
requested_networks)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 571, in _allocate_network<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp
requested_networks=requested_networks)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/network/api.py", line
178, in allocate_for_instance<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp 'args': args})<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/__init__.py",
line 68, in call<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp return
_get_impl().call(context, topic, msg, timeout)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 674, in call<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp return
rpc_amqp.call(context, topic, msg, timeout, Connection.pool)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line
343, in call<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp rv = list(rv)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line
304, in __iter__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self.done()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp self.gen.next()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line
301, in __iter__<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp
self._iterator.next()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 572, in iterconsume<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp yield
self.ensure(_error_callback, _consume)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 503, in ensure<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp
error_callback(e)<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py",
line 553, in _error_callback<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp raise
rpc_common.Timeout()<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp Timeout: Timeout
while waiting on RPC response.<br>
2013-02-25 18:21:28 TRACE nova.rpc.amqp<br>
<br>
Thanks in advance,<br>
<br>
Javi<br>
<br>
-- <br>
Javier Álvarez Cid-Fuentes<br>
Grid Computing and Clusters Group<br>
Barcelona Supercomputing Center (BSC-CNS)<br>
Tel. (+34) 93 413 72 46<br>
<br>
<br>
WARNING / LEGAL TEXT: This message is intended only for the
use of the<br>
individual or entity to which it is addressed and may
contain<br>
information which is privileged, confidential, proprietary,
or exempt<br>
from disclosure under applicable law. If you are not the
intended<br>
recipient or the person responsible for delivering the
message to the<br>
intended recipient, you are strictly prohibited from
disclosing,<br>
distributing, copying, or in any way using this message. If
you have<br>
received this communication in error, please notify the
sender and<br>
destroy and delete any copies you may have received.<br>
<br>
<a href="http://www.bsc.es/disclaimer" target="_blank">http://www.bsc.es/disclaimer</a><br>
<br>
_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<br>
<pre cols="72">--
Javier Álvarez Cid-Fuentes
Grid Computing and Clusters Group
Barcelona Supercomputing Center (BSC-CNS)
Tel. (+34) 93 413 72 46</pre>
<br>
<br>
WARNING / LEGAL TEXT: This message is intended only for the use of the
individual or entity to which it is addressed and may contain
information which is privileged, confidential, proprietary, or exempt
from disclosure under applicable law. If you are not the intended
recipient or the person responsible for delivering the message to the
intended recipient, you are strictly prohibited from disclosing,
distributing, copying, or in any way using this message. If you have
received this communication in error, please notify the sender and
destroy and delete any copies you may have received.
<br><br>
<a href="http://www.bsc.es/disclaimer.htm" target="_blank">http://www.bsc.es/disclaimer</a>
<br>
</div></div></div>
<br>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div></div>