After fixing the routing_ip in the nova.conf file, my instances can now connect to the outside world. However, i'm still unable to connect to any of my instances through ssh or ping from the controller node. The controller node is also running a nova-network, could that be the issue?<br>
<br><div class="gmail_quote">On Wed, Sep 5, 2012 at 4:36 PM, George Mihaiescu <span dir="ltr"><<a href="mailto:George.Mihaiescu@q9.com" target="_blank">George.Mihaiescu@q9.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<u></u>
<div lang="EN-US" link="blue" vlink="blue">
<div>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy">No, if you run in a multi-host enabled
mode there is no need for ip_forward enabled on the controller node.<u></u><u></u></span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy">I would check iptables on the nova-compute
hosts, arp tables in the instance, outside connectivity from the hosts, etc.<u></u><u></u></span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy"><u></u> <u></u></span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy"><u></u> <u></u></span></font></p>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">
<hr size="2" width="100%" align="center">
</span></font></div>
<p class="MsoNormal"><b><font face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma;font-weight:bold">From:</span></font></b><font face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma"> Leander Bessa
Beernaert [mailto:<a href="mailto:leanderbb@gmail.com" target="_blank">leanderbb@gmail.com</a>] <br>
<b><span style="font-weight:bold">Sent:</span></b> Wednesday, September 05,
2012 11:26 AM<br>
<b><span style="font-weight:bold">To:</span></b> <u></u>George
Mihaiescu</span></font></p><div><div class="h5"><font face="Tahoma"><u></u><br>
<b><span style="font-weight:bold">Cc:</span></b> <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
<b><span style="font-weight:bold">Subject:</span></b> Re: [Openstack]
[OpenStack][Nova]Problems and questions regarding network and/or routing</font></div></div><u></u><u></u><p></p>
</div><div><div class="h5">
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><br>
</span></font><font color="#222222" face="Arial"><span style="font-size:11.0pt;font-family:Arial;color:#222222;background:white">Do i also need
ip_forward enabled on the controller node? (sorry about the duplicate)</span></font><u></u><u></u></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Wed, Sep 5, 2012 at 4:25 PM, Leander Bessa Beernaert <<a href="mailto:leanderbb@gmail.com" target="_blank">leanderbb@gmail.com</a>>
wrote:<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Do i also no ip_forward enabled on the controller node?<u></u><u></u></span></font></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Wed, Sep 5, 2012 at 3:18 PM, Leander Bessa Beernaert <<a href="mailto:leanderbb@gmail.com" target="_blank">leanderbb@gmail.com</a>>
wrote:<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">All compute nodes have /proc/sys/net/ipv4/ip_forward set to 1. So
that can't be the issue :s<u></u><u></u></span></font></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Wed, Sep 5, 2012 at 3:11 PM, <u></u>George
Mihaiescu<u></u> <<a href="mailto:George.Mihaiescu@q9.com" target="_blank">George.Mihaiescu@q9.com</a>> wrote:<u></u><u></u></span></font></p>
<div link="blue" vlink="blue">
<div>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy">Hi Leander,</span></font><u></u><u></u></p>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font><u></u><u></u></p>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy">Make sure you have ip forward enabled on the nova-compute nodes
(that now act as nova-network as well).</span></font><u></u><u></u></p>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy">Second, each nova-network acts as a gateway for each project and it
needs an IP address, so probably this explains the “phantom” </span></font>10.0.108.4,
10.0.108.6, 10.0.108.8 and 10.0.108.4.10 addresses.<u></u><u></u></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy">George</span></font><u></u><u></u></p>
<p class="MsoNormal"><font color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font><u></u><u></u></p>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">
<hr size="2" width="100%" align="center">
</span></font></div>
<p class="MsoNormal"><b><font face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma;font-weight:bold">From:</span></font></b><font face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma"> openstack-bounces+george.mihaiescu=<a href="mailto:q9.com@lists.launchpad.net" target="_blank">q9.com@lists.launchpad.net</a>
[mailto:<a href="mailto:openstack-bounces%2Bgeorge.mihaiescu" target="_blank">openstack-bounces+george.mihaiescu</a>=<a href="mailto:q9.com@lists.launchpad.net" target="_blank">q9.com@lists.launchpad.net</a>]
<b><span style="font-weight:bold">On Behalf Of </span></b>Leander Bessa
Beernaert<br>
<b><span style="font-weight:bold">Sent:</span></b> Wednesday, September 05,
2012 9:49 AM<br>
<b><span style="font-weight:bold">To:</span></b> Vishvananda Ishaya<br>
<b><span style="font-weight:bold">Cc:</span></b> <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
<b><span style="font-weight:bold">Subject:</span></b> Re: [Openstack]
[OpenStack][Nova]Problems and questions regarding network and/or routing</span></font><u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">I'm having the
strangest issue. I have set up a separate OpenStack cluster to test out the
multi-host setup. <u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">I have one
controller node and 4 compute nodes. Each compute node is running nova-network,
nova-compute and nova-api-metadata. I have set up a tenant with the a
multi-host network on the address range <a href="http://10.0.108.0/24" target="_blank">10.0.108.0/24</a>. <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">I launched 4
instances to fill up the compute nodes:<u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<div>
<p class="MsoNormal"><font size="3" face="Courier New"><span style="font-size:12.0pt;font-family:"Courier New"">+--------------------------------------+------+--------+----------------------------+</span></font><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Courier New"><span style="font-size:12.0pt;font-family:"Courier New"">|
ID
| Name | Status |
Networks |</span></font><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Courier New"><span style="font-size:12.0pt;font-family:"Courier New"">+--------------------------------------+------+--------+----------------------------+</span></font><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Courier New"><span style="font-size:12.0pt;font-family:"Courier New"">|
2c63cc3e-7c45-4e10-8ac5-480fa60d4f32 | Test | ACTIVE |
project_network=10.0.108.7 |</span></font><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Courier New"><span style="font-size:12.0pt;font-family:"Courier New"">|
c48f6aae-0d97-4e69-a398-8cda929c310d | Test | ACTIVE |
project_network=10.0.108.3 |</span></font><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Courier New"><span style="font-size:12.0pt;font-family:"Courier New"">|
ed8f11a4-5fc0-4437-9ae2-b6725126fca7 | Test | ACTIVE |
project_network=10.0.108.5 |</span></font><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Courier New"><span style="font-size:12.0pt;font-family:"Courier New"">|
fe39e586-030c-4bf4-9020-7ef773567913 | Test | ACTIVE |
project_network=10.0.108.9 |</span></font><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Courier New"><span style="font-size:12.0pt;font-family:"Courier New"">+--------------------------------------+------+--------+----------------------------+</span></font><u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">One thing i found
odd at the beginning was the fact that the instances are using only odd
addresses. <u></u>The<u></u> installation is clean
and no instances have been launched before, so all the addresses are
available. <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><u></u><font size="3" face="Times New Roman"><span style="font-size:12.0pt">The</span></font><u></u>
problem now is that i can only ping instances form the compute nodes. I am
unable to ping any instance from the controller node. Stranger yet, is the fact
that i can ping non-existent address such as 10.0.108.4,10.0.108.6,10.0.108.8
and 10.0.108.4.10. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">I have also no
connectivity from within the instances to the outside world.<u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Has this happend
to anyone before?<u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Tue, Sep 4,
2012 at 11:38 PM, Vishvananda Ishaya <<a href="mailto:vishvananda@gmail.com" target="_blank">vishvananda@gmail.com</a>> wrote:<u></u><u></u></span></font></p>
<div style="word-wrap:break-word">
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
<div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Sep 4, 2012, at
3:01 PM, Leander Bessa Beernaert <<a href="mailto:leanderbb@gmail.com" target="_blank">leanderbb@gmail.com</a>> wrote:<u></u><u></u></span></font></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Question follows
inlined below.<u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Tue, Sep 4,
2012 at 6:48 PM, Vishvananda Ishaya <<a href="mailto:vishvananda@gmail.com" target="_blank">vishvananda@gmail.com</a>> wrote:<u></u><u></u></span></font></p>
<div style="word-wrap:break-word">
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
<div>
<div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Sep 4, 2012, at
8:35 AM, Leander Bessa Beernaert <<a href="mailto:leanderbb@gmail.com" target="_blank">leanderbb@gmail.com</a>> wrote:<u></u><u></u></span></font></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Hello all,<u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">I've had a few
reports from users testing out the sample installation of OpenStack i setup.
<u></u>The<u></u> reports were all related to
problems with inter-vm network speeds and connection timeouts as well as the
inability to connect to the outside word from within the VM (e.g.: ping <a href="http://www.google.com/" target="_blank">www.google.com</a>). I'm not
sure if i setup something wrong, so i have a few questions.<u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><u></u><font size="3" face="Times New Roman"><span style="font-size:12.0pt">The</span></font><u></u>
current installation of OpenStack is running with 1 controller node,
and 8 compute nodes. Each node is running Ubuntu 12.04 and Essex with the
default packages. I'm using the VLAN network manager. <u></u>The<u></u>re is one peculiarity to this setup. Since each
physical hosts only has 1 network interface, i came up with the
following configuration:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> - For
inter-node communications i set up a VLAN with the ID 107<u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> - Each
tentant has it's private network on a separate VLAN. Currently there are two
tenant, one on VLAN 109 (network: <a href="http://10.0.9.0/24" target="_blank">10.0.9.0/24</a>)
and another on VLAN 110 (network: <a href="http://10.0.9.0/24" target="_blank">10.0.9.0/24</a>). <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">I'm not a network
expert, so please bear with me if i make any outrages statements. <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">1) When
communicating on the private network, the packets are not routed through the
controller right? That only happens when the VM needs to contact an external
source (e.g.: google), correct? This report originated from users from
VLAN 109. <u></u>The<u></u>y are using network
intensive applications which send a lot of data between each of the instances. <u></u>The<u></u>y reported various time-out and connection drops
as well as slow transfer speeds. I'm no network expert, but could this be
related to the routing, VLANs or is it a hardware issue?<u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
</div>
<p class="MsoNormal"><u></u><font size="3" face="Times New Roman"><span style="font-size:12.0pt">The</span></font><u></u>re
are a lot of things that could cause this. You would need to do some extensive
debugging to find the source of this.<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Any ideas where i
can start looking? <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Also,
communications between two VMs on different compute nodes from the same tenant
do not need to be routed through the controller node right?<u></u><u></u></span></font></p>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">in non-multi_host
mode I believe it will go through the controller.<u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Vish<u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><br>
<br clear="all">
<u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">-- <u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Cumprimentos /
Regards,<u></u><u></u></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Leander<u></u><u></u></span></font></p>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><br>
<br clear="all">
<u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">-- <u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Cumprimentos / Regards,<u></u><u></u></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Leander<u></u><u></u></span></font></p>
</div>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><br>
<br clear="all">
<u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">-- <u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Cumprimentos / Regards,<u></u><u></u></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Leander<u></u><u></u></span></font></p>
</div>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><br>
<br clear="all">
<u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">-- <u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Cumprimentos / Regards,<u></u><u></u></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Leander<u></u><u></u></span></font></p>
</div></div></div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div>Cumprimentos / Regards,</div>Leander<br>