[Openstack-operators] Problem starting image on addtional nova compute (raise rpc_common.Timeout())
Salman Toor
salman.toor at it.uu.se
Mon Sep 24 12:01:56 UTC 2012
Hi,
I want to start an instance on a newly deployed compute node. But got
following error messages.
------------ host: openstack-com, compute.log ----------------------------
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._publish_service_capabilities from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:43:07 DEBUG nova.manager [-] Notifying Schedulers of
capabilities ... from (pid=5116) _publish_service_capabilities
/usr/lib/python2.6/site-packages/nova/manager.py:203
2012-09-24 13:43:07 DEBUG nova.rpc.amqp [-] Making asynchronous fanout
cast... from (pid=5116) fanout_cast
/usr/lib/python2.6/site-packages/nova/rpc/amqp.py:359
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_rescued_instances from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._sync_power_states from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_bandwidth_usage from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager.update_available_resource from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:43:07 INFO nova.virt.libvirt.connection [-]
Compute_service record updated for openstack-com.cern.ch
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_rebooting_instances from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:43:07 DEBUG nova.manager [-] Skipping
ComputeManager._cleanup_running_deleted_instances, 18 ticks left until
next run from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:147
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._heal_instance_info_cache from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:43:07 DEBUG nova.manager [-] Skipping
ComputeManager._run_image_cache_manager_pass, 38 ticks left until next
run from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:147
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._reclaim_queued_deletes from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:43:07 DEBUG nova.compute.manager [-]
FLAGS.reclaim_instance_interval <= 0, skipping... from (pid=5116)
_reclaim_queued_deletes
/usr/lib/python2.6/site-packages/nova/compute/manager.py:2404
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._report_driver_status from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:43:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_unconfirmed_resizes from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._publish_service_capabilities from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:07 DEBUG nova.manager [-] Notifying Schedulers of
capabilities ... from (pid=5116) _publish_service_capabilities
/usr/lib/python2.6/site-packages/nova/manager.py:203
2012-09-24 13:44:07 DEBUG nova.rpc.amqp [-] Making asynchronous fanout
cast... from (pid=5116) fanout_cast
/usr/lib/python2.6/site-packages/nova/rpc/amqp.py:359
2012-09-24 13:44:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_rescued_instances from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:07 DEBUG nova.manager [-] Skipping
ComputeManager._sync_power_states, 10 ticks left until next run from
(pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:147
2012-09-24 13:44:07 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_bandwidth_usage from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:07 DEBUG nova.manager [-] Running periodic task
ComputeManager.update_available_resource from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:08 INFO nova.virt.libvirt.connection [-]
Compute_service record updated for openstack-com.cern.ch
2012-09-24 13:44:08 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_rebooting_instances from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:08 DEBUG nova.manager [-] Skipping
ComputeManager._cleanup_running_deleted_instances, 17 ticks left until
next run from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:147
2012-09-24 13:44:08 DEBUG nova.manager [-] Running periodic task
ComputeManager._heal_instance_info_cache from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:08 DEBUG nova.manager [-] Skipping
ComputeManager._run_image_cache_manager_pass, 37 ticks left until next
run from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:147
2012-09-24 13:44:08 DEBUG nova.manager [-] Running periodic task
ComputeManager._reclaim_queued_deletes from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:08 DEBUG nova.compute.manager [-]
FLAGS.reclaim_instance_interval <= 0, skipping... from (pid=5116)
_reclaim_queued_deletes
/usr/lib/python2.6/site-packages/nova/compute/manager.py:2404
2012-09-24 13:44:08 DEBUG nova.manager [-] Running periodic task
ComputeManager._report_driver_status from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:08 INFO nova.compute.manager [-] Updating host status
2012-09-24 13:44:08 DEBUG nova.virt.libvirt.connection [-] Updating
host stats from (pid=5116) update_status
/usr/lib/python2.6/site-packages/nova/virt/libvirt/connection.py:2500
2012-09-24 13:44:08 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_unconfirmed_resizes from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:44:18 DEBUG nova.rpc.amqp [-] received
{u'_context_roles': [u'admin'], u'_context_request_id':
u'req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c', u'_context_read_deleted':
u'no', u'args': {u'instance_uuid':
u'47c3e22d-75f1-4fd3-95aa-2fa3db0f666a', u'requested_networks': None,
u'is_first_time': True, u'admin_password': '<SANITIZED>',
u'injected_files': []}, u'_context_auth_token': '<SANITIZED>',
u'_context_is_admin': True, u'_context_project_id':
u'461226cb29194c86b84964896e2fefbc', u'_context_timestamp':
u'2012-09-24T11:44:18.004777', u'_context_user_id':
u'91fa17b603c74c238e89d3b334bb0ceb', u'method': u'run_instance',
u'_context_remote_address': u'188.184.20.238'} from (pid=5116)
_safe_log /usr/lib/python2.6/site-packages/nova/rpc/common.py:160
2012-09-24 13:44:18 DEBUG nova.rpc.amqp
[req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c
91fa17b603c74c238e89d3b334bb0ceb 461226cb29194c86b84964896e2fefbc]
unpacked context: {'user_id': u'91fa17b603c74c238e89d3b334bb0ceb',
'roles': [u'admin'], 'timestamp': '2012-09-24T11:44:18.004777',
'auth_token': '<SANITIZED>', 'remote_address': u'188.184.20.238',
'is_admin': True, 'request_id':
u'req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c', 'project_id':
u'461226cb29194c86b84964896e2fefbc', 'read_deleted': u'no'} from
(pid=5116) _safe_log
/usr/lib/python2.6/site-packages/nova/rpc/common.py:160
2012-09-24 13:44:18 DEBUG nova.utils
[req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c
91fa17b603c74c238e89d3b334bb0ceb 461226cb29194c86b84964896e2fefbc]
Attempting to grab semaphore "47c3e22d-75f1-4fd3-95aa-2fa3db0f666a"
for method "do_run_instance"... from (pid=5116) inner
/usr/lib/python2.6/site-packages/nova/utils.py:928
2012-09-24 13:44:18 DEBUG nova.utils
[req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c
91fa17b603c74c238e89d3b334bb0ceb 461226cb29194c86b84964896e2fefbc] Got
semaphore "47c3e22d-75f1-4fd3-95aa-2fa3db0f666a" for method
"do_run_instance"... from (pid=5116) inner
/usr/lib/python2.6/site-packages/nova/utils.py:932
2012-09-24 13:44:18 AUDIT nova.compute.manager
[req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c
91fa17b603c74c238e89d3b334bb0ceb 461226cb29194c86b84964896e2fefbc]
[instance: 47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] Starting instance...
2012-09-24 13:44:19 DEBUG nova.rpc.amqp
[req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c
91fa17b603c74c238e89d3b334bb0ceb 461226cb29194c86b84964896e2fefbc]
Making asynchronous call on network ... from (pid=5116) multicall
/usr/lib/python2.6/site-packages/nova/rpc/amqp.py:326
2012-09-24 13:44:19 DEBUG nova.rpc.amqp
[req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c
91fa17b603c74c238e89d3b334bb0ceb 461226cb29194c86b84964896e2fefbc]
MSG_ID is 55e70c2bf3084c4b880b025130e33295 from (pid=5116) multicall
/usr/lib/python2.6/site-packages/nova/rpc/amqp.py:329
2012-09-24 13:45:08 DEBUG nova.manager [-] Running periodic task
ComputeManager._publish_service_capabilities from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:45:08 DEBUG nova.manager [-] Notifying Schedulers of
capabilities ... from (pid=5116) _publish_service_capabilities
/usr/lib/python2.6/site-packages/nova/manager.py:203
2012-09-24 13:45:08 DEBUG nova.rpc.amqp [-] Making asynchronous fanout
cast... from (pid=5116) fanout_cast
/usr/lib/python2.6/site-packages/nova/rpc/amqp.py:359
2012-09-24 13:45:08 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_rescued_instances from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:45:08 DEBUG nova.manager [-] Skipping
ComputeManager._sync_power_states, 9 ticks left until next run from
(pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:147
2012-09-24 13:45:08 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_bandwidth_usage from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:45:08 DEBUG nova.manager [-] Running periodic task
ComputeManager.update_available_resource from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:45:09 INFO nova.virt.libvirt.connection [-]
Compute_service record updated for openstack-com.cern.ch
2012-09-24 13:45:09 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_rebooting_instances from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:45:09 DEBUG nova.manager [-] Skipping
ComputeManager._cleanup_running_deleted_instances, 16 ticks left until
next run from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:147
2012-09-24 13:45:09 DEBUG nova.manager [-] Running periodic task
ComputeManager._heal_instance_info_cache from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:45:09 DEBUG nova.rpc.amqp [-] Making asynchronous call
on network ... from (pid=5116) multicall
/usr/lib/python2.6/site-packages/nova/rpc/amqp.py:326
2012-09-24 13:45:09 DEBUG nova.rpc.amqp [-] MSG_ID is
3702e883e2e8421f839332004922333f from (pid=5116) multicall
/usr/lib/python2.6/site-packages/nova/rpc/amqp.py:329
2012-09-24 13:45:09 DEBUG nova.compute.manager [-] Updated the
info_cache for instance 47c3e22d-75f1-4fd3-95aa-2fa3db0f666a from
(pid=5116) _heal_instance_info_cache
/usr/lib/python2.6/site-packages/nova/compute/manager.py:2251
2012-09-24 13:45:09 DEBUG nova.manager [-] Skipping
ComputeManager._run_image_cache_manager_pass, 36 ticks left until next
run from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:147
2012-09-24 13:45:09 DEBUG nova.manager [-] Running periodic task
ComputeManager._reclaim_queued_deletes from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:45:09 DEBUG nova.compute.manager [-]
FLAGS.reclaim_instance_interval <= 0, skipping... from (pid=5116)
_reclaim_queued_deletes
/usr/lib/python2.6/site-packages/nova/compute/manager.py:2404
2012-09-24 13:45:09 DEBUG nova.manager [-] Running periodic task
ComputeManager._report_driver_status from (pid=5116) periodic_tasks
/usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:45:09 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_unconfirmed_resizes from (pid=5116)
periodic_tasks /usr/lib/python2.6/site-packages/nova/manager.py:152
2012-09-24 13:45:19 ERROR nova.rpc.impl_qpid
[req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c
91fa17b603c74c238e89d3b334bb0ceb 461226cb29194c86b84964896e2fefbc]
Timed out waiting for RPC response: None
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid Traceback (most recent
call last):
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 364, in
ensure
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid return method(*args,
**kwargs)
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 413, in
_consume
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid nxt_receiver =
self.session.next_receiver(timeout=timeout)
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid File "<string>", line
6, in next_receiver
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid File
"/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line
651, in next_receiver
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid raise Empty
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid Empty: None
2012-09-24 13:45:19 TRACE nova.rpc.impl_qpid
2012-09-24 13:45:19 ERROR nova.compute.manager
[req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c
91fa17b603c74c238e89d3b334bb0ceb 461226cb29194c86b84964896e2fefbc]
[instance: 47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] Instance failed
network setup
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] Traceback (most recent call last):
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 578,
in _allocate_network
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a]
requested_networks=requested_networks)
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/network/api.py", line 178, in
allocate_for_instance
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] 'args': args})
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/rpc/__init__.py", line 68, in
call
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] return
_get_impl().call(context, topic, msg, timeout)
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 526, in
call
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] return
rpc_amqp.call(context, topic, msg, timeout, Connection.pool)
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/rpc/amqp.py", line 343, in call
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] rv = list(rv)
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/rpc/amqp.py", line 304, in
__iter__
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] self.done()
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] self.gen.next()
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/rpc/amqp.py", line 301, in
__iter__
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] self._iterator.next()
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 422, in
iterconsume
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] yield
self.ensure(_error_callback, _consume)
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 368, in
ensure
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] error_callback(e)
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 407, in
_error_callback
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] raise rpc_common.Timeout()
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a] Timeout: Timeout while waiting
on RPC response.
2012-09-24 13:45:19 TRACE nova.compute.manager [instance:
47c3e22d-75f1-4fd3-95aa-2fa3db0f666a]
2012-09-24 13:45:19 ERROR nova.rpc.amqp
[req-7a85d8e9-73d2-4b99-aa52-5544ac29b96c
91fa17b603c74c238e89d3b334bb0ceb 461226cb29194c86b84964896e2fefbc]
Exception during message handling
2012-09-24 13:45:19 TRACE nova.rpc.amqp Traceback (most recent call last):
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/rpc/amqp.py", line 253, in
_process_data
2012-09-24 13:45:19 TRACE nova.rpc.amqp rval =
node_func(context=ctxt, **node_args)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/exception.py", line 114, in
wrapped
2012-09-24 13:45:19 TRACE nova.rpc.amqp return f(*args, **kw)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 183,
in decorated_function
2012-09-24 13:45:19 TRACE nova.rpc.amqp sys.exc_info())
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2012-09-24 13:45:19 TRACE nova.rpc.amqp self.gen.next()
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 177,
in decorated_function
2012-09-24 13:45:19 TRACE nova.rpc.amqp return function(self,
context, instance_uuid, *args, **kwargs)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 675,
in run_instance
2012-09-24 13:45:19 TRACE nova.rpc.amqp do_run_instance()
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/utils.py", line 946, in inner
2012-09-24 13:45:19 TRACE nova.rpc.amqp retval = f(*args, **kwargs)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 674,
in do_run_instance
2012-09-24 13:45:19 TRACE nova.rpc.amqp
self._run_instance(context, instance_uuid, **kwargs)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 470,
in _run_instance
2012-09-24 13:45:19 TRACE nova.rpc.amqp
self._set_instance_error_state(context, instance_uuid)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2012-09-24 13:45:19 TRACE nova.rpc.amqp self.gen.next()
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 443,
in _run_instance
2012-09-24 13:45:19 TRACE nova.rpc.amqp requested_networks)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 578,
in _allocate_network
2012-09-24 13:45:19 TRACE nova.rpc.amqp
requested_networks=requested_networks)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/network/api.py", line 178, in
allocate_for_instance
2012-09-24 13:45:19 TRACE nova.rpc.amqp 'args': args})
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/rpc/__init__.py", line 68, in
call
2012-09-24 13:45:19 TRACE nova.rpc.amqp return
_get_impl().call(context, topic, msg, timeout)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 526, in
call
2012-09-24 13:45:19 TRACE nova.rpc.amqp return
rpc_amqp.call(context, topic, msg, timeout, Connection.pool)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/rpc/amqp.py", line 343, in call
2012-09-24 13:45:19 TRACE nova.rpc.amqp rv = list(rv)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/rpc/amqp.py", line 304, in
__iter__
2012-09-24 13:45:19 TRACE nova.rpc.amqp self.done()
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2012-09-24 13:45:19 TRACE nova.rpc.amqp self.gen.next()
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/rpc/amqp.py", line 301, in
__iter__
2012-09-24 13:45:19 TRACE nova.rpc.amqp self._iterator.next()
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 422, in
iterconsume
2012-09-24 13:45:19 TRACE nova.rpc.amqp yield
self.ensure(_error_callback, _consume)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 368, in
ensure
2012-09-24 13:45:19 TRACE nova.rpc.amqp error_callback(e)
2012-09-24 13:45:19 TRACE nova.rpc.amqp File
"/usr/lib/python2.6/site-packages/nova/rpc/impl_qpid.py", line 407, in
_error_callback
2012-09-24 13:45:19 TRACE nova.rpc.amqp raise rpc_common.Timeout()
2012-09-24 13:45:19 TRACE nova.rpc.amqp Timeout: Timeout while waiting
on RPC response.
service status:
Binary Host Zone
Status State Updated_At
nova-compute openstack-con.cern.ch nova
enabled XXX 2012-09-24 09:35:59
nova-network openstack-con.cern.ch nova
enabled :-) 2012-09-24 11:47:23
nova-scheduler openstack-con.cern.ch nova
enabled :-) 2012-09-24 11:47:21
nova-volume openstack-con.cern.ch nova
enabled :-) 2012-09-24 11:47:16
nova-cert openstack-con.cern.ch nova
enabled :-) 2012-09-24 11:47:20
nova-consoleauth openstack-con.cern.ch nova
enabled :-) 2012-09-24 11:47:16
nova-compute openstack-com.cern.ch nova
enabled :-) 2012-09-24 11:47:22
nova-cert openstack-com.cern.ch nova
enabled :-) 2012-09-24 11:47:23
nova-console openstack-con.cern.ch nova
enabled :-) 2012-09-24 11:47:21
in order to clear the test deployment
1 - openstack-con, Running as a controller node. running network,
compute. glance, keystone, volume, api, cert, consoleauth,
objectstore, libvertd and qpidd
2 - openstack-com, Running as a Compute node. api, cert, compute and libvertd
Do I really need to run api and cert on each compute node?
It is clear that its something related to messaging. But if the
problem in communication than how come openstack-com communicates with
openstack-con network service.....
qpidd.conf
----
cluster-mechanism=DIGEST-MD5 ANONYMOUS
auth=no
----
Please give me some hint.
Regards..
Salman.
More information about the OpenStack-operators
mailing list