<div dir="ltr"><div><div><div><div><div>Hi Dan,<br><br></div>Thanks for the reply. I agree to your point of using supported Distributed Virtual Switch plugin for ESX <br>rather than going on with standard vSwitch of ESX.<br>
<br></div>Currently,
we are using the Grizzly release and have KVM with Openvswitch. We had
the requirement of integrating ESX as well with the current setup. As we
know that support for multiple neutron plugins is not there in Grizzly,
hence we opted of having a workaround to see if we could use
openvswitch and obtain the same functionality.<br><br></div>Today we were able to achieve the end-to-end flow of traffic by adding rules manually to the openvswitch-switches in nova-compute vm. If support for the configuring flows in switches is added through API's, maybe we can support openvswitch as well. Though, ideally one should not use vSwitch as its having minimal capabilities and one should always go with the DVS for ESX.<br>
<br></div>-Regards<br></div>Rahul Sharma <br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Sep 12, 2013 at 10:16 PM, Dan Wendlandt <span dir="ltr"><<a href="mailto:dan@nicira.com" target="_blank">dan@nicira.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Rahul,<div><br></div><div>Thanks for the detailed description of your setup.</div><div><br></div><div>
>From my understanding of your diagram, you are trying to mix and match two incompatible mechanisms: ESX networking and the OVS Neutron plugin with GRE tunneling. </div>
<div><br></div><div>If you're just trying to get something simple working with ESX, you can use basic nova-networking with ESX. Otherwise, I'd suggest you check out a compatible Neutron plugin (see the compatibility list here: <a href="http://docs.openstack.org/trunk/openstack-network/admin/content/flexibility.html" target="_blank">http://docs.openstack.org/trunk/openstack-network/admin/content/flexibility.html</a> )</div>
<div> <br></div><div><br></div><div>Dan</div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Sep 12, 2013 at 4:48 AM, Rahul Sharma <span dir="ltr"><<a href="mailto:rahulsharmaait@gmail.com" target="_blank">rahulsharmaait@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div link="blue" vlink="purple" lang="EN-US"><div><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""></span>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi All,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">When we create port-group “br-int” on ESX and launch instance, instance gets launched on ESX and is assigned port-group br-int. Since this br-int is unable
to communicate with network-node over GRE, communication fails. Diagram with “initial-setup” shown below lists the connectivity of Nova-compute placed on ESX-host and instances getting launched on ESX host:-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><img src="cid:image001.png@01CEAFDB.11F0DEA0" height="582" width="662"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">To allow vm’s to communicate with network node over GRE, we can assign one more nic(eth2) to nova-compute, put br-int(esx) in promiscuous mode and add eth2
to “br-int” on nova-compute. Now the packet will traverse as VM -> br-int(esx) -> eth2(compute) -> br-int(compute) -> br-tun(compute) -> Network-Node(over GRE tunnel). Below diagram explains the same:-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><img src="cid:image002.png@01CEAFDB.11F0DEA0" height="584" width="653"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Still this will not work because the rules configured on openvswitches (br-int and br-tun) will drop the packets!!!<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Inbuilt Openvswitch-controller configures the vswitches to allow only specific flows which matches the rules installed on them. Even if we add eth2 to br-int,
we will also need to add generic rules to br-int and br-tun such that they are able to pass the packets received from eth2 to br-int, then to br-tun and then to network node over GRE tunnel. Here is one sample output of the flow-dumps of br-int and br-tun
of compute node:-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">br-int flows:-<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">NXST_FLOW reply (xid=0x4):<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d;background:lime">cookie=0x0, duration=96.138s, table=0, n_packets=0, n_bytes=0, priority=1 actions=NORMAL</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">br-tun flows:-<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">NXST_FLOW reply (xid=0x4):<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d;background:red">cookie=0x0, duration=98.322s, table=0, n_packets=0, n_bytes=0, priority=1 actions=drop</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Can someone help me in identifying what flows I should add such that I am not breaking any functionality of quantum. Though the above workaround will allow
vm’s on ESX to communicate with one another which should not be allowed(if they are under different tenants), rest everything almost works fine.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Any inputs or suggestions for this would be greatly acknowledged.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks and Regards<span><font color="#888888"><u></u><u></u></font></span></span></p>
<span><font color="#888888">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Rahul Sharma<u></u><u></u></span></p></font></span></div></div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br>~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>Dan Wendlandt <div>Nicira, Inc: <a href="http://www.nicira.com" target="_blank">www.nicira.com</a><br>
<div>twitter: danwendlandt<br>
~~~~~~~~~~~~~~~~~~~~~~~~~~~<br></div></div>
</font></span></div>
<br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br></div>