<div dir="ltr">Hi Rahul,<div><br></div><div>Thanks for the detailed description of your setup.</div><div><br></div><div>From my understanding of your diagram, you are trying to mix and match two incompatible mechanisms: ESX networking and the OVS Neutron plugin with GRE tunneling. </div>
<div><br></div><div style>If you're just trying to get something simple working with ESX, you can use basic nova-networking with ESX. Otherwise, I'd suggest you check out a compatible Neutron plugin (see the compatibility list here: <a href="http://docs.openstack.org/trunk/openstack-network/admin/content/flexibility.html">http://docs.openstack.org/trunk/openstack-network/admin/content/flexibility.html</a> )</div>
<div style> <br></div><div style><br></div><div style>Dan</div><div style><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Sep 12, 2013 at 4:48 AM, Rahul Sharma <span dir="ltr"><<a href="mailto:rahulsharmaait@gmail.com" target="_blank">rahulsharmaait@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div link="blue" vlink="purple" lang="EN-US"><div><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""></span>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi All,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">When we create port-group “br-int” on ESX and launch instance, instance gets launched on ESX and is assigned port-group br-int. Since this br-int is unable
to communicate with network-node over GRE, communication fails. Diagram with “initial-setup” shown below lists the connectivity of Nova-compute placed on ESX-host and instances getting launched on ESX host:-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><img src="cid:image001.png@01CEAFDB.11F0DEA0" height="582" width="662"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">To allow vm’s to communicate with network node over GRE, we can assign one more nic(eth2) to nova-compute, put br-int(esx) in promiscuous mode and add eth2
to “br-int” on nova-compute. Now the packet will traverse as VM -> br-int(esx) -> eth2(compute) -> br-int(compute) -> br-tun(compute) -> Network-Node(over GRE tunnel). Below diagram explains the same:-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><img src="cid:image002.png@01CEAFDB.11F0DEA0" height="584" width="653"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Still this will not work because the rules configured on openvswitches (br-int and br-tun) will drop the packets!!!<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Inbuilt Openvswitch-controller configures the vswitches to allow only specific flows which matches the rules installed on them. Even if we add eth2 to br-int,
we will also need to add generic rules to br-int and br-tun such that they are able to pass the packets received from eth2 to br-int, then to br-tun and then to network node over GRE tunnel. Here is one sample output of the flow-dumps of br-int and br-tun
of compute node:-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">br-int flows:-<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">NXST_FLOW reply (xid=0x4):<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d;background:lime">cookie=0x0, duration=96.138s, table=0, n_packets=0, n_bytes=0, priority=1 actions=NORMAL</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">br-tun flows:-<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">NXST_FLOW reply (xid=0x4):<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d;background:red">cookie=0x0, duration=98.322s, table=0, n_packets=0, n_bytes=0, priority=1 actions=drop</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Can someone help me in identifying what flows I should add such that I am not breaking any functionality of quantum. Though the above workaround will allow
vm’s on ESX to communicate with one another which should not be allowed(if they are under different tenants), rest everything almost works fine.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Any inputs or suggestions for this would be greatly acknowledged.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks and Regards<span class="HOEnZb"><font color="#888888"><u></u><u></u></font></span></span></p>
<span class="HOEnZb"><font color="#888888">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Rahul Sharma<u></u><u></u></span></p></font></span></div></div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>Dan Wendlandt <div>Nicira, Inc: <a href="http://www.nicira.com" target="_blank">www.nicira.com</a><br><div>twitter: danwendlandt<br>
~~~~~~~~~~~~~~~~~~~~~~~~~~~<br></div></div>
</div>