<p dir="ltr">Setup from scratch will be the safest, but just so you know, multi_host = 1 is probably the most common deployment these days. To make it work just run nova-compute nova-network and nova-api-metadata on each compute node.</p>
<div class="gmail_quote">On Nov 20, 2012 5:23 PM, "Ahmed Al-Mehdi" <<a href="mailto:ahmed@coraid.com">ahmed@coraid.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word"><div><div>Hi Vish,</div><div><br></div><div>Thank you very much gain. I never set multi_host=True in my nova.conf file. But I think I know how it could have got set. In the "OpenStack Install and Deploy Manual", section "Creating the Network for Compute VMs" ( <a href="http://docs.openstack.org/folsom/openstack-compute/install/apt/content/compute-create-network.html" target="_blank">http://docs.openstack.org/folsom/openstack-compute/install/apt/content/compute-create-network.html</a> ), I used the following command from the section to create the network:</div>
<div><pre><span style="font-size:9pt;font-family:Courier"><span style="white-space:pre-wrap"> </span>nova-manage network create private <b>--multi_host=T </b>--fixed_range_v4=<a href="http://192.168.100.0/24" target="_blank">192.168.100.0/24</a> --bridge_interface=br100 --num_networks=1 --network_size=256
</span></pre></div><div><b>Is that a typo in the document?</b></div><div><br></div><div>I would really prefer to run "multi_host=0" mode, so I don't run into other issues. I am not familiar with, so I feel I might end up doing more damage than good trying to muck with the db. What would you suggest, run nova-network on sonoma, and that will pretty much solve my issues, or should I just re-do the setup from scratch, which would not be too bad for me as it will take me an hour or two.</div>
<div><br></div><div>Regards,</div><div>Ahmed.</div></div><div><br></div><div><br></div><span><div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span> Vishvananda Ishaya <<a href="mailto:vishvananda@gmail.com" target="_blank">vishvananda@gmail.com</a>><br><span style="font-weight:bold">Date: </span> Tuesday, November 20, 2012 4:59 PM<br>
<span style="font-weight:bold">To: </span> Ahmed Al-Mehdi <<a href="mailto:ahmed@coraid.com" target="_blank">ahmed@coraid.com</a>><br><span style="font-weight:bold">Cc: </span> "<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>" <<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>><br>
<span style="font-weight:bold">Subject: </span> Re: [Openstack] Floating IP vs. Fixed IP in nova-network<br></div><div><br></div><blockquote style="BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5"><div><div style="word-wrap:break-word">
Your hosts get ips from the fixed range in multi_host mode. It looks like your network is multi_host=True. this is the only reason sonoma would have been assigned an ip. So you can either run nova-network on sonoma or set multi_host=0 on your network in the
database. If you want to switch back to multi_host=0 you will likely have to clean out the db tables, so you might do best dropping all records from the networks and fixed_ips tables and recreating the network.
<div><br></div><div>Vish</div><div><br><div><div>On Nov 20, 2012, at 4:52 PM, Ahmed Al-Mehdi <<a href="mailto:ahmed@coraid.com" target="_blank">ahmed@coraid.com</a>> wrote:</div><br><blockquote type="cite"><div style="word-wrap:break-word;font-size:14px;font-family:Calibri,sans-serif">
<div>Hi Vish,</div><div><br></div><div>Thank you very much for your help, I really appreciate it.</div><div><br></div><div>My setup has two nodes:</div><div><br></div><div>controller-node (hostname: bodega; nova-network running; no nova-compute running)</div>
<div><div><span style="white-space:pre-wrap"></span>eth0:<span style="white-space:pre-wrap"></span>10.176.20.158</div><div><span style="white-space:pre-wrap"></span>eth1:<span style="white-space:pre-wrap"></span>No IP assigned (VM network)</div>
<div> </div></div><div>compute-node (hostname: sonoma; nova-compute running only). </div><div><div><span style="white-space:pre-wrap"></span>eth0:<span style="white-space:pre-wrap"></span>10.176.20.4</div><div><span style="white-space:pre-wrap"></span>eth1:<span style="white-space:pre-wrap"></span>No IP assigned (VM network)</div>
</div><div><br></div><div>My network configuration is in single-host mode. Both the host's hostname and their IPs has not changed.</div><div><br></div><div>I believe my setup is affected with issue (a), "have a network or fixed ip with an old hostname assigned".</div>
<div><br></div><div><div>root@bodega:/etc/nova# mysql -u root -pmysqlsecret nova -e 'select * from fixed_ips where host="sonoma"'</div><div>+---------------------+---------------------+------------+---------+----+---------------+------------+-----------+--------+----------+----------------------+--------+---------------+</div>
<div>| created_at | updated_at | deleted_at | deleted | id | address | network_id | allocated | leased | reserved | virtual_interface_id | host | instance_uuid |</div><div>+---------------------+---------------------+------------+---------+----+---------------+------------+-----------+--------+----------+----------------------+--------+---------------+</div>
<div>| 2012-11-13 18:49:37 | 2012-11-16 21:45:32 | NULL | 0 | 3 | 192.168.100.2 | 1 | 0 | 0 | 0 | NULL | sonoma | NULL |</div><div>+---------------------+---------------------+------------+---------+----+---------------+------------+-----------+--------+----------+----------------------+--------+---------------+</div>
<div>root@bodega:/etc/nova# </div></div><div><div>root@bodega:/etc/nova# mysql -u root -pmysqlsecret nova -e 'select * from networks where host="sonoma"'</div><div>root@bodega:/etc/nova# (NO OUTPUT)</div>
</div><div><br></div><div>I am a bit confused, why is "192.168.100.2" assigned to sonoma? Isn't that IP range reserved for VMs?</div><div><br></div><div>Should sonoma have the IP address "10.176.20.4"? How can I clear the issue, so I don't get the RPC message timeout.</div>
<div><br></div><div><br></div><div>Regards,</div><div>Ahmed.</div><div><br></div><div><br></div><span><div style="font-family:Calibri;font-size:11pt;text-align:left;border-width:1pt medium medium;border-style:solid none none;padding:3pt 0in 0in;border-top-color:rgb(181,196,223)">
<span style="font-weight:bold">From: </span>Vishvananda Ishaya <<a href="mailto:vishvananda@gmail.com" target="_blank">vishvananda@gmail.com</a>><br><span style="font-weight:bold">Date: </span>Tuesday, November 20, 2012 4:08 PM<br>
<span style="font-weight:bold">To: </span>Ahmed Al-Mehdi <<a href="mailto:ahmed@coraid.com" target="_blank">ahmed@coraid.com</a>><br><span style="font-weight:bold">Cc: </span>"<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>" <<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Openstack] Floating IP vs. Fixed IP in nova-network<br></div><div><br></div><blockquote style="BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5" type="cite"><div style="word-wrap:break-word">
<br><div><div>On Nov 20, 2012, at 4:02 PM, Ahmed Al-Mehdi <<a href="mailto:ahmed@coraid.com" target="_blank">ahmed@coraid.com</a>> wrote:</div><br><blockquote type="cite"><div style="word-wrap:break-word;font-size:14px;font-family:Calibri,sans-serif">
<div>Hi Vish,</div><div><br></div><div>I do not have auto_assign<span style="font-style:italic">_</span>floating_ip set. So, I can safely assume nova-network is assigning fixed-IP, right?</div></div></blockquote><div><br>
</div>
you always get a fixed ip<br><blockquote type="cite"><div style="word-wrap:break-word;font-size:14px;font-family:Calibri,sans-serif"><div><br></div><div>Sorry to impose, but do you have a few minutes to help me understand by I am getting a RPC message timeout issue which is prohibiting me from launching a VM. I looked through the logs extensively, but I can't figure out, who the RPC msg is destine for,
and why no response.</div></div></blockquote><div><br></div>
do you have a machine with the hostname sonoma? Perhaps you did at one point and the hostname has changed?</div><div><br></div><div>I suspect either:</div><div><br></div><div>a) you have a network or fixed ip with an old hostname assigned:</div>
<div>mysql nova -e 'select * from fixed_ips where host="sonoma"'</div><div>mysql nova -e 'select * from networks where host="sonoma"'</div><div><br></div><div>or (more likely)</div><div>
b) you are running multi_host mode and you have nova-compute running on the host 'sonoma' and you don't have nova-network running on the host like you should.</div><div><br></div><div>Vish</div><div><br><blockquote type="cite">
<div style="word-wrap:break-word;font-size:14px;font-family:Calibri,sans-serif"><div><br></div><div><div>2012-11-18 15:50:29 DEBUG nova.openstack.common.rpc.amqp [-] <b>Making asynchronous call on network.sonoma</b> ... from (pid=1375) multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp</div>
<div>.py:351</div><div>2012-11-18 15:50:29 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is d73be9ea76b3412493d0752abb9d5a02 from (pid=1375) multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:</div>
<div>354</div><div>2012-11-18 15:50:52 DEBUG nova.openstack.common.rpc.amqp [-] received {u'_context_roles': [], u'_msg_id': u'b2bc0715982846cd916a8ff61b2513af', u'_context_quota_class': None, u'_context_request_id':</div>
<div> u'req-22e6e99a-c582-449c-8d61-d4ee57f1ac57', u'_context_service_catalog': None, u'_context_user_name': None, u'_context_auth_token': '<SANITIZED>', u'args': {u'instance_id': 5, u'instance_uuid': u</div>
<div>'4e80964e-5bd1-4df4-a517-223c79d55517', u'host': u'sonoma', u'project_id': u'ce1e819636744dc680fa5515f6475e87', u'rxtx_factor': 1.0}, u'_context_instance_lock_checked': False, u'_context_project_na</div>
<div>me': None, u'_context_is_admin': True, u'_context_project_id': None, u'_context_timestamp': u'2012-11-18T23:50:47.233052', u'_context_read_deleted': u'no', u'_context_user_id': None, u'method': u'g</div>
<div>et_instance_nw_info', u'_context_remote_address': None} from (pid=1375) _safe_log /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py:195</div><div>2012-11-18 15:50:52 DEBUG nova.openstack.common.rpc.amqp [-] unpacked context: {'project_name': None, 'user_id': None, 'roles': [], 'timestamp': u'2012-11-18T23:50:47.233052', 'auth_token': '<SANIT</div>
<div>IZED>', 'remote_address': None, 'quota_class': None, 'is_admin': True, 'service_catalog': None, 'request_id': u'req-22e6e99a-c582-449c-8d61-d4ee57f1ac57', 'instance_lock_checked': False, 'project_i</div>
<div>d': None, 'user_name': None, 'read_deleted': u'no'} from (pid=1375) _safe_log /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py:195</div><div>2012-11-18 15:50:52 DEBUG nova.utils [req-22e6e99a-c582-449c-8d61-d4ee57f1ac57 None None] Got semaphore "get_dhcp" for method "_get_dhcp_ip"... from (pid=1375) inner /usr/lib/python2.7/dist-package</div>
<div>s/nova/utils.py:713</div><div>2012-11-18 15:50:52 DEBUG nova.utils [req-22e6e99a-c582-449c-8d61-d4ee57f1ac57 None None] Got semaphore "get_dhcp" for method "_get_dhcp_ip"... from (pid=1375) inner /usr/lib/python2.7/dist-package</div>
<div>s/nova/utils.py:713</div><div>2012-11-18 15:51:09 DEBUG nova.manager [-] Running periodic task FlatDHCPManager._publish_service_capabilities from (pid=1375) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:172</div>
<div>2012-11-18 15:51:09 DEBUG nova.manager [-] Running periodic task FlatDHCPManager._disassociate_stale_fixed_ips from (pid=1375) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:172</div><div>2012-11-18 15:51:29 ERROR nova.openstack.common.rpc.common [-] <b>Timed out waiting for RPC response: timed out</b></div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common Traceback (most recent call last):</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 513, in ensure</div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return method(*args, **kwargs)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 590, in _consume</div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 175, in drain_events</div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 238, in drain_events</div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return connection.drain_events(**kwargs)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 57, in drain_events</div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.wait_multi(self.channels.values(), timeout=timeout)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 63, in wait_multi</div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common chanmap.keys(), allowed_methods, timeout=timeout)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 120, in _wait_multiple</div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common channel, method_sig, args, content = read_timeout(timeout)</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 94, in read_timeout</div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common return self.method_reader.read_method()</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/method_framing.py", line 221, in read_method</div>
<div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common raise m</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common timeout: timed out</div><div>2012-11-18 15:51:29 TRACE nova.openstack.common.rpc.common</div>
</div><div><br></div><div>Thank you,</div><div>Ahmed.</div></div></blockquote></div><br></div></blockquote></span></div></blockquote></div><br></div></div></div></blockquote></span></div>
</blockquote></div>