<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; "><div>Hello,</div><div><br></div><div>I have a two node OpenStack setup, one control-node and one compute node, with the services up and running on each node.</div><div><br></div><div><div>After a minute of issuing the "nova-boot" command on the control-node, I see the following error message in nova-network.log file</div><div> <span class="Apple-tab-span" style="white-space: pre; "> ERROR nova.openstack.common.rpc.common [-] Timed out waiting for RPC response: timed out</span></div></div><div><br></div><div>I feel like I am running into some networking configuration issue. Can someone who is familiar with nova-network please help.</div><div><br></div><div><br></div><div>control-node (hostname: bodega, eth0 : 10.176.20.158, eth1 : internal, VM network)</div><div>compute-node (hostname: sonoma, eth0: 10.176.20.4, eth1: internal, VM network)</div><div><br></div><div><div>root@bodega:~# nova-manage service list</div><div>Binary Host Zone Status State Updated_At</div><div>nova-network bodega nova enabled :-) 2012-11-19 00:27:17</div><div>nova-scheduler bodega nova enabled :-) 2012-11-19 00:27:22</div><div>nova-compute sonoma nova enabled :-) 2012-11-19 00:27:21</div><div>nova-cert <span class="Apple-tab-span" style="white-space:pre"> </span>bodega nova enabled :-) 2012-11-19 00:27:13</div><div>nova-consoleauth bodega nova enabled :-) 2012-11-19 00:27:12</div></div><div><br></div><div><br></div><div><div>root@bodega:~# date; nova boot --flavor 2 --image 19beac6b-5a46-4712-a5b5-99c6e99daa67 --key_name mykey --security_group default cirros</div><div>Sun Nov 18 15:50:26 PST 2012</div><div>+-------------------------------------+----------------------------------------------------------+</div><div>| Property | Value |</div><div>+-------------------------------------+----------------------------------------------------------+</div><div>| OS-DCF:diskConfig | MANUAL |</div><div>| OS-EXT-SRV-ATTR:host | sonoma |</div><div>| OS-EXT-SRV-ATTR:hypervisor_hostname | sonoma.eng-rwc.coraid.com |</div><div>| OS-EXT-SRV-ATTR:instance_name | instance-00000005 |</div><div>| OS-EXT-STS:power_state | 0 |</div><div>| OS-EXT-STS:task_state | scheduling |</div><div>| OS-EXT-STS:vm_state | building |</div><div>| accessIPv4 | |</div><div>| accessIPv6 | |</div><div>| adminPass | Zk6VS8U7Kdoe |</div><div>| config_drive | |</div><div>| created | 2012-11-18T23:50:28Z |</div><div>| flavor | m1.small |</div><div>| hostId | af229f25f97dbd0b697da3d76d8f6fc09cf36a36a181b43ecf065843 |</div><div>| id | 4e80964e-5bd1-4df4-a517-223c79d55517 |</div><div>| image | cirros-0.3.0-x86_64 |</div><div>| key_name | mykey |</div><div>| metadata | {} |</div><div>| name | cirros |</div><div>| progress | 0 |</div><div>| security_groups | [{u'name': u'default'}] |</div><div>| status | BUILD |</div><div>| tenant_id | ce1e819636744dc680fa5515f6475e87 |</div><div>| updated | 2012-11-18T23:50:28Z |</div><div>| user_id | ce016bb05df949ebbafcc7c165359d7c |</div><div>+-------------------------------------+----------------------------------------------------------+</div><div>root@bodega:~# nova list</div><div>+--------------------------------------+--------+--------+----------+</div><div>| ID | Name | Status | Networks |</div><div>+--------------------------------------+--------+--------+----------+</div><div>| 4e80964e-5bd1-4df4-a517-223c79d55517 | cirros | BUILD | |</div><div>+--------------------------------------+--------+--------+----------+</div><div>root@bodega:~# sleep 5; date; nova list</div><div>Sun Nov 18 15:51:30 PST 2012</div><div>+--------------------------------------+--------+--------+----------+</div><div>| ID | Name | Status | Networks |</div><div>+--------------------------------------+--------+--------+----------+</div><div>| 4e80964e-5bd1-4df4-a517-223c79d55517 | cirros | ERROR | |</div><div>+--------------------------------------+--------+--------+----------+</div></div><div><br></div><div><br></div><div><div>nova-network.log</div><div><br></div><div>2012-11-18 15:51:29 ERROR nova.openstack.common.rpc.common [-] Timed out waiting for RPC response: timed out</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common Traceback (most recent call last):</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 513, in ensure</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return method(*args, **kwargs)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 590, in _consume</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 175, in drain_events</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 238, in drain_events</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return connection.drain_events(**kwargs)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 57, in drain_events</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.wait_multi(self.channels.values(), timeout=timeout)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 63, in wait_multi</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common chanmap.keys(), allowed_methods, timeout=timeout)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 120, in _wait_multiple</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 94, in read_timeout</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.method_reader.read_method()</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/method_framing.py", line 221, in read_method</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common raise m</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common timeout: timed out</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common</div><div>2012-11-18 15:51:29 DEBUG nova.utils [req-f2b4df1f-6c29-48dc-991b-93fb5eb29d08 ce016bb05df949ebbafcc7c165359d7c ce1e819636744dc680fa5515f6475e87] Got semaphore "get_dhcp" for method "_get_dhcp_ip"...</div><div>from (pid=1375) inner /usr/lib/python2.7/dist-packages/nova/utils.py:713</div><div>2012-11-18 15:51:29 DEBUG nova.utils [req-f2b4df1f-6c29-48dc-991b-93fb5eb29d08 ce016bb05df949ebbafcc7c165359d7c ce1e819636744dc680fa5515f6475e87] Got semaphore "get_dhcp" for method "_get_dhcp_ip"...</div><div>from (pid=1375) inner /usr/lib/python2.7/dist-packages/nova/utils.py:713</div><div>2012-11-18 15:51:58 DEBUG nova.openstack.common.rpc.amqp [-] received {u'_context_roles': [], u'_msg_id': u'c1537de718aa4678b51cdb7c9628e502', u'_context_quota_class': None, u'_context_request_id': u'</div><div>req-b54a9c72-edc8-4013-8095-1af3a078c6ac', u'_context_service_catalog': None, u'_context_user_name': None, u'_context_auth_token': '<SANITIZED>', u'args': {u'instance_id': 5, u'instance_uuid': u'4e809</div><div>64e-5bd1-4df4-a517-223c79d55517', u'host': u'sonoma', u'project_id': u'ce1e819636744dc680fa5515f6475e87', u'rxtx_factor': 1.0}, u'_context_instance_lock_checked': False, u'_context_project_name': None</div><div>, u'_context_is_admin': True, u'_context_project_id': None, u'_context_timestamp': u'2012-11-18T23:51:52.661432', u'_context_read_deleted': u'no', u'_context_user_id': None, u'method': u'get_instance_</div><div>nw_info', u'_context_remote_address': None} from (pid=1375) _safe_log /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py:195</div><div>2012-11-18 15:51:58 DEBUG nova.openstack.common.rpc.amqp [-] unpacked context: {'project_name': None, 'user_id': None, 'roles': [], 'timestamp': u'2012-11-18T23:51:52.661432', 'auth_token': '<SANITIZE</div><div>D>', 'remote_address': None, 'quota_class': None, 'is_admin': True, 'service_catalog': None, 'request_id': u'req-b54a9c72-edc8-4013-8095-1af3a078c6ac', 'instance_lock_checked': False, 'project_id': No</div><div>ne, 'user_name': None, 'read_deleted': u'no'} from (pid=1375) _safe_log /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py:195</div><div>2012-11-18 15:51:58 DEBUG nova.utils [req-b54a9c72-edc8-4013-8095-1af3a078c6ac None None] Got semaphore "get_dhcp" for method "_get_dhcp_ip"... from (pid=1375) inner /usr/lib/python2.7/dist-packages/n</div><div>ova/utils.py:713</div><div>2012-11-18 15:51:58 DEBUG nova.utils [req-b54a9c72-edc8-4013-8095-1af3a078c6ac None None] Got semaphore "get_dhcp" for method "_get_dhcp_ip"... from (pid=1375) inner /usr/lib/python2.7/dist-packages/n</div><div>ova/utils.py:713</div><div><br></div></div><div>Thank you,</div><div>Ahmed.</div><div><br></div></body></html>