[Openstack] VM instance creation fails due to nova-network encountering an RPC timeout.
Ahmed Al-Mehdi
ahmed at coraid.com
Mon Nov 19 00:40:40 UTC 2012
Hello,
I have a two node OpenStack setup, one control-node and one compute node, with the services up and running on each node.
After a minute of issuing the "nova-boot" command on the control-node, I see the following error message in nova-network.log file
ERROR nova.openstack.common.rpc.common [-] Timed out waiting for RPC response: timed out
I feel like I am running into some networking configuration issue. Can someone who is familiar with nova-network please help.
control-node (hostname: bodega, eth0 : 10.176.20.158, eth1 : internal, VM network)
compute-node (hostname: sonoma, eth0: 10.176.20.4, eth1: internal, VM network)
root at bodega:~# nova-manage service list
Binary Host Zone Status State Updated_At
nova-network bodega nova enabled :-) 2012-11-19 00:27:17
nova-scheduler bodega nova enabled :-) 2012-11-19 00:27:22
nova-compute sonoma nova enabled :-) 2012-11-19 00:27:21
nova-cert bodega nova enabled :-) 2012-11-19 00:27:13
nova-consoleauth bodega nova enabled :-) 2012-11-19 00:27:12
root at bodega:~# date; nova boot --flavor 2 --image 19beac6b-5a46-4712-a5b5-99c6e99daa67 --key_name mykey --security_group default cirros
Sun Nov 18 15:50:26 PST 2012
+-------------------------------------+----------------------------------------------------------+
| Property | Value |
+-------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-SRV-ATTR:host | sonoma |
| OS-EXT-SRV-ATTR:hypervisor_hostname | sonoma.eng-rwc.coraid.com |
| OS-EXT-SRV-ATTR:instance_name | instance-00000005 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | Zk6VS8U7Kdoe |
| config_drive | |
| created | 2012-11-18T23:50:28Z |
| flavor | m1.small |
| hostId | af229f25f97dbd0b697da3d76d8f6fc09cf36a36a181b43ecf065843 |
| id | 4e80964e-5bd1-4df4-a517-223c79d55517 |
| image | cirros-0.3.0-x86_64 |
| key_name | mykey |
| metadata | {} |
| name | cirros |
| progress | 0 |
| security_groups | [{u'name': u'default'}] |
| status | BUILD |
| tenant_id | ce1e819636744dc680fa5515f6475e87 |
| updated | 2012-11-18T23:50:28Z |
| user_id | ce016bb05df949ebbafcc7c165359d7c |
+-------------------------------------+----------------------------------------------------------+
root at bodega:~# nova list
+--------------------------------------+--------+--------+----------+
| ID | Name | Status | Networks |
+--------------------------------------+--------+--------+----------+
| 4e80964e-5bd1-4df4-a517-223c79d55517 | cirros | BUILD | |
+--------------------------------------+--------+--------+----------+
root at bodega:~# sleep 5; date; nova list
Sun Nov 18 15:51:30 PST 2012
+--------------------------------------+--------+--------+----------+
| ID | Name | Status | Networks |
+--------------------------------------+--------+--------+----------+
| 4e80964e-5bd1-4df4-a517-223c79d55517 | cirros | ERROR | |
+--------------------------------------+--------+--------+----------+
nova-network.log
2012-11-18 15:51:29 ERROR nova.openstack.common.rpc.common [-] Timed out waiting for RPC response: timed out
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common Traceback (most recent call last):
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 513, in ensure
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return method(*args, **kwargs)
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 590, in _consume
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout)
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 175, in drain_events
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs)
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 238, in drain_events
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return connection.drain_events(**kwargs)
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 57, in drain_events
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.wait_multi(self.channels.values(), timeout=timeout)
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 63, in wait_multi
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common chanmap.keys(), allowed_methods, timeout=timeout)
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 120, in _wait_multiple
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout)
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 94, in read_timeout
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.method_reader.read_method()
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/method_framing.py", line 221, in read_method
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common raise m
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common timeout: timed out
2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common
2012-11-18 15:51:29 DEBUG nova.utils [req-f2b4df1f-6c29-48dc-991b-93fb5eb29d08 ce016bb05df949ebbafcc7c165359d7c ce1e819636744dc680fa5515f6475e87] Got semaphore "get_dhcp" for method "_get_dhcp_ip"...
from (pid=1375) inner /usr/lib/python2.7/dist-packages/nova/utils.py:713
2012-11-18 15:51:29 DEBUG nova.utils [req-f2b4df1f-6c29-48dc-991b-93fb5eb29d08 ce016bb05df949ebbafcc7c165359d7c ce1e819636744dc680fa5515f6475e87] Got semaphore "get_dhcp" for method "_get_dhcp_ip"...
from (pid=1375) inner /usr/lib/python2.7/dist-packages/nova/utils.py:713
2012-11-18 15:51:58 DEBUG nova.openstack.common.rpc.amqp [-] received {u'_context_roles': [], u'_msg_id': u'c1537de718aa4678b51cdb7c9628e502', u'_context_quota_class': None, u'_context_request_id': u'
req-b54a9c72-edc8-4013-8095-1af3a078c6ac', u'_context_service_catalog': None, u'_context_user_name': None, u'_context_auth_token': '<SANITIZED>', u'args': {u'instance_id': 5, u'instance_uuid': u'4e809
64e-5bd1-4df4-a517-223c79d55517', u'host': u'sonoma', u'project_id': u'ce1e819636744dc680fa5515f6475e87', u'rxtx_factor': 1.0}, u'_context_instance_lock_checked': False, u'_context_project_name': None
, u'_context_is_admin': True, u'_context_project_id': None, u'_context_timestamp': u'2012-11-18T23:51:52.661432', u'_context_read_deleted': u'no', u'_context_user_id': None, u'method': u'get_instance_
nw_info', u'_context_remote_address': None} from (pid=1375) _safe_log /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py:195
2012-11-18 15:51:58 DEBUG nova.openstack.common.rpc.amqp [-] unpacked context: {'project_name': None, 'user_id': None, 'roles': [], 'timestamp': u'2012-11-18T23:51:52.661432', 'auth_token': '<SANITIZE
D>', 'remote_address': None, 'quota_class': None, 'is_admin': True, 'service_catalog': None, 'request_id': u'req-b54a9c72-edc8-4013-8095-1af3a078c6ac', 'instance_lock_checked': False, 'project_id': No
ne, 'user_name': None, 'read_deleted': u'no'} from (pid=1375) _safe_log /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py:195
2012-11-18 15:51:58 DEBUG nova.utils [req-b54a9c72-edc8-4013-8095-1af3a078c6ac None None] Got semaphore "get_dhcp" for method "_get_dhcp_ip"... from (pid=1375) inner /usr/lib/python2.7/dist-packages/n
ova/utils.py:713
2012-11-18 15:51:58 DEBUG nova.utils [req-b54a9c72-edc8-4013-8095-1af3a078c6ac None None] Got semaphore "get_dhcp" for method "_get_dhcp_ip"... from (pid=1375) inner /usr/lib/python2.7/dist-packages/n
ova/utils.py:713
Thank you,
Ahmed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20121118/ca3021e9/attachment.html>
More information about the Openstack
mailing list