Open Stack

Hello all,

We are testing a fresh installation of icehouse un ubuntu 14.04 following
the official docs.

The problem: on certain reboots of our compute nodes, we get the stack
trace below when the instances try to obtain their metadata from
nova-api-metadata, and timeout errors appear in the instance's cloud-init
logs. After several minutes  the errors disappear.

We've checked rabbitmq connectivity and it's ok; after the compute node is
running, everything works ok, ie. we can reboot instances without errors,
so... It seems to be a boot process specific error, and it's not raised on
all the reboots.

Any clue where to look now?
Thanks!


nova-api-metadata.log:

TRACE nova.api.metadata.handler Traceback (most recent call last):
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/nova/api/metadata/handler.py", line 132,
in _handle_remote_ip_request
TRACE nova.api.metadata.handler     meta_data =
self.get_metadata_by_remote_address(remote_address)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/nova/api/metadata/handler.py", line 74,
in get_metadata_by_remote_address
TRACE nova.api.metadata.handler     data =
base.get_metadata_by_address(self.conductor_api, address)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/nova/api/metadata/base.py", line 507, in
get_metadata_by_address
TRACE nova.api.metadata.handler     ctxt)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/nova/api/metadata/base.py", line 514, in
get_metadata_by_instance_id
TRACE nova.api.metadata.handler     return InstanceMetadata(instance,
address)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/nova/api/metadata/base.py", line 165, in
__init__
TRACE nova.api.metadata.handler     instance)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/nova/network/api.py", line 94, in wrapped
TRACE nova.api.metadata.handler     return func(self, context, *args,
**kwargs)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/nova/network/api.py", line 389, in
get_instance_nw_info
TRACE nova.api.metadata.handler     result =
self._get_instance_nw_info(context, instance)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/nova/network/api.py", line 405, in
_get_instance_nw_info
TRACE nova.api.metadata.handler     nw_info =
self.network_rpcapi.get_instance_nw_info(context, **args)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line 222, in
get_instance_nw_info
TRACE nova.api.metadata.handler     host=host, project_id=project_id)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py", line 150,
in call
TRACE nova.api.metadata.handler     wait_for_reply=True, timeout=timeout)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/oslo/messaging/transport.py", line 90, in
_send
TRACE nova.api.metadata.handler     timeout=timeout)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 412, in send
TRACE nova.api.metadata.handler     return self._send(target, ctxt,
message, wait_for_reply, timeout)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 403, in _send
TRACE nova.api.metadata.handler     result = self._waiter.wait(msg_id,
timeout)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 267, in wait
TRACE nova.api.metadata.handler     reply, ending =
self._poll_connection(msg_id, timeout)
TRACE nova.api.metadata.handler   File
"/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 217, in _poll_connection
TRACE nova.api.metadata.handler     % msg_id)
TRACE nova.api.metadata.handler MessagingTimeout: Timed out waiting for a
reply to message ID f9fb930ba84b491e886e746f3df788a5
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20140619/8da5c3e3/attachment.html>