Open Stack

Hi:

Have you ensured your messaging queue is working correctly?

Regards,
JuanFra


2013/2/25 Javier Alvarez <javier.alvarez at bsc.es>

> Hello,
>
> I'm trying to setup OpenStack Essex on Debian and I'm getting an error
> when creating new VMs. The problem arises when trying to allocate the
> network of the VM, but I'm not sure why it's happening, I'm using
> nova-network, and that's the log output:
>
> 2013-02-25 18:20:28 AUDIT nova.compute.manager
> [req-4d211944-bd5c-4e88-a8df-**c4a9167676e2 36a7e60b4d134307b61e06949d3373
> **5e 9558f53959a04cc992c3f8b6d91bfb**9f] [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60] Starting instance...
> 2013-02-25 18:20:40 INFO nova.virt.libvirt.connection [-] Compute_service
> record updated for bscgrid21
> 2013-02-25 18:20:40 INFO nova.compute.manager [-] Updating host status
> 2013-02-25 18:21:28 ERROR nova.rpc.common [req-4d211944-bd5c-4e88-a8df-**c4a9167676e2
> 36a7e60b4d134307b61e06949d3373**5e 9558f53959a04cc992c3f8b6d91bfb**9f]
> Timed out waiting for RPC response: timed out
> 2013-02-25 18:21:28 TRACE nova.rpc.common Traceback (most recent call
> last):
> 2013-02-25 18:21:28 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-
> **packages/nova/rpc/impl_kombu.**py", line 490, in ensure
> 2013-02-25 18:21:28 TRACE nova.rpc.common     return method(*args,
> **kwargs)
> 2013-02-25 18:21:28 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-
> **packages/nova/rpc/impl_kombu.**py", line 567, in _consume
> 2013-02-25 18:21:28 TRACE nova.rpc.common     return
> self.connection.drain_events(**timeout=timeout)
> 2013-02-25 18:21:28 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-
> **packages/kombu/connection.py", line 167, in drain_events
> 2013-02-25 18:21:28 TRACE nova.rpc.common     return
> self.transport.drain_events(**self.connection, **kwargs)
> 2013-02-25 18:21:28 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-
> **packages/kombu/transport/**amqplib.py", line 262, in drain_events
> 2013-02-25 18:21:28 TRACE nova.rpc.common     return
> connection.drain_events(****kwargs)
> 2013-02-25 18:21:28 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-
> **packages/kombu/transport/**amqplib.py", line 94, in drain_events
> 2013-02-25 18:21:28 TRACE nova.rpc.common     return
> self.wait_multi(self.channels.**values(), timeout=timeout)
> 2013-02-25 18:21:28 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-
> **packages/kombu/transport/**amqplib.py", line 100, in wait_multi
> 2013-02-25 18:21:28 TRACE nova.rpc.common     chanmap.keys(),
> allowed_methods, timeout=timeout)
> 2013-02-25 18:21:28 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-
> **packages/kombu/transport/**amqplib.py", line 159, in _wait_multiple
> 2013-02-25 18:21:28 TRACE nova.rpc.common     channel, method_sig, args,
> content = read_timeout(timeout)
> 2013-02-25 18:21:28 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-
> **packages/kombu/transport/**amqplib.py", line 132, in read_timeout
> 2013-02-25 18:21:28 TRACE nova.rpc.common     return
> self.method_reader.read_**method()
> 2013-02-25 18:21:28 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-
> **packages/amqplib/client_0_8/**method_framing.py", line 221, in
> read_method
> 2013-02-25 18:21:28 TRACE nova.rpc.common     raise m
> 2013-02-25 18:21:28 TRACE nova.rpc.common timeout: timed out
> 2013-02-25 18:21:28 TRACE nova.rpc.common
> 2013-02-25 18:21:28 ERROR nova.compute.manager
> [req-4d211944-bd5c-4e88-a8df-**c4a9167676e2 36a7e60b4d134307b61e06949d3373
> **5e 9558f53959a04cc992c3f8b6d91bfb**9f] [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60] Instance failed network setup
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60] Traceback (most recent call last):
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**
> packages/nova/compute/manager.**py", line 571, in _allocate_network
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60] requested_networks=requested_**
> networks)
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**packages/nova/network/api.py",
> line 178, in allocate_for_instance
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     'args': args})
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/__init__.py"**, line 68, in call
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     return
> _get_impl().call(context, topic, msg, timeout)
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/impl_kombu.**py", line 674, in call
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     return rpc_amqp.call(context,
> topic, msg, timeout, Connection.pool)
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**packages/nova/rpc/amqp.py",
> line 343, in call
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     rv = list(rv)
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**packages/nova/rpc/amqp.py",
> line 304, in __iter__
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     self.done()
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/**contextlib.py",
> line 24, in __exit__
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     self.gen.next()
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**packages/nova/rpc/amqp.py",
> line 301, in __iter__
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     self._iterator.next()
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/impl_kombu.**py", line 572, in iterconsume
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     yield
> self.ensure(_error_callback, _consume)
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/impl_kombu.**py", line 503, in ensure
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     error_callback(e)
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/impl_kombu.**py", line 553, in _error_callback
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]     raise rpc_common.Timeout()
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60] Timeout: Timeout while waiting on
> RPC response.
> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
> 8c5b2d4a-c715-4f4d-8d74-**a6bdccbe1b60]
> 2013-02-25 18:21:28 ERROR nova.rpc.amqp [req-4d211944-bd5c-4e88-a8df-**c4a9167676e2
> 36a7e60b4d134307b61e06949d3373**5e 9558f53959a04cc992c3f8b6d91bfb**9f]
> Exception during message handling
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp Traceback (most recent call last):
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**packages/nova/rpc/amqp.py",
> line 253, in _process_data
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     rval = node_func(context=ctxt,
> **node_args)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**packages/nova/exception.py",
> line 114, in wrapped
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     return f(*args, **kw)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/compute/manager.**py", line 183, in decorated_function
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     sys.exc_info())
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/**contextlib.py",
> line 24, in __exit__
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     self.gen.next()
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/compute/manager.**py", line 177, in decorated_function
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     return function(self, context,
> instance_uuid, *args, **kwargs)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/compute/manager.**py", line 663, in run_instance
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     do_run_instance()
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**packages/nova/utils.py",
> line 945, in inner
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     retval = f(*args, **kwargs)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/compute/manager.**py", line 662, in do_run_instance
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp self._run_instance(context,
> instance_uuid, **kwargs)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/compute/manager.**py", line 463, in _run_instance
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp self._set_instance_error_**state(context,
> instance_uuid)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/**contextlib.py",
> line 24, in __exit__
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     self.gen.next()
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/compute/manager.**py", line 436, in _run_instance
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     requested_networks)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/compute/manager.**py", line 571, in _allocate_network
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp requested_networks=requested_**
> networks)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**packages/nova/network/api.py",
> line 178, in allocate_for_instance
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     'args': args})
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/__init__.py"**, line 68, in call
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     return
> _get_impl().call(context, topic, msg, timeout)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/impl_kombu.**py", line 674, in call
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     return rpc_amqp.call(context,
> topic, msg, timeout, Connection.pool)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**packages/nova/rpc/amqp.py",
> line 343, in call
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     rv = list(rv)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**packages/nova/rpc/amqp.py",
> line 304, in __iter__
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     self.done()
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/**contextlib.py",
> line 24, in __exit__
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     self.gen.next()
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**packages/nova/rpc/amqp.py",
> line 301, in __iter__
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     self._iterator.next()
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/impl_kombu.**py", line 572, in iterconsume
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     yield
> self.ensure(_error_callback, _consume)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/impl_kombu.**py", line 503, in ensure
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     error_callback(e)
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/dist-**
> packages/nova/rpc/impl_kombu.**py", line 553, in _error_callback
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp     raise rpc_common.Timeout()
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp Timeout: Timeout while waiting on
> RPC response.
> 2013-02-25 18:21:28 TRACE nova.rpc.amqp
>
> Thanks in advance,
>
> Javi
>
> --
> Javier Álvarez Cid-Fuentes
> Grid Computing and Clusters Group
> Barcelona Supercomputing Center (BSC-CNS)
> Tel. (+34) 93 413 72 46
>
>
> WARNING / LEGAL TEXT: This message is intended only for the use of the
> individual or entity to which it is addressed and may contain
> information which is privileged, confidential, proprietary, or exempt
> from disclosure under applicable law. If you are not the intended
> recipient or the person responsible for delivering the message to the
> intended recipient, you are strictly prohibited from disclosing,
> distributing, copying, or in any way using this message. If you have
> received this communication in error, please notify the sender and
> destroy and delete any copies you may have received.
>
> http://www.bsc.es/disclaimer
>
> ______________________________**_________________
> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> Post to     : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> More help   : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20130225/7e35ca44/attachment.html>