<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin-top:0in;
margin-right:0in;
margin-bottom:10.0pt;
margin-left:0in;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:10.0pt;
margin-left:.5in;
mso-add-space:auto;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
mso-add-space:auto;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
mso-add-space:auto;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:10.0pt;
margin-left:.5in;
mso-add-space:auto;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1737317399;
mso-list-type:hybrid;
mso-list-template-ids:-1321561128 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal">Here is an update on the issue of not being able to capture packets flowing between two VMs on the same host.<o:p></o:p></p>
<p class="MsoNormal"><u>Isolating the problem:<o:p></o:p></u></p>
<p class="MsoNormal">It has been confirmed that the TaaS OVS driver cannot capture packets ingressing (flowing into) a VM’s vNIC under the following conditions:<o:p></o:p></p>
<p class="MsoNormal">- The sender port and the receiver port (which is being monitored) reside on the same host.<o:p></o:p></p>
<p class="MsoNormal">- The sender port and the receiver port are attached to the same (virtual) network.<o:p></o:p></p>
<p class="MsoNormal">We do not have a problem in the above situation for traffic egressing (flowing out) of the VM’s vNIC. We also do not have a problem if the sender port and receiver port are on different hosts, or if the sender and receiver ports belong
to different (virtual) networks.<o:p></o:p></p>
<p class="MsoNormal"><u>Trafffic Capture Technique:<o:p></o:p></u></p>
<p class="MsoNormal">The implementation uses the following methods to capture packets flowing into and out of a VM’s vNIC.<o:p></o:p></p>
<p class="MsoNormal">Egress Side (flowing out of the vNIC):<o:p></o:p></p>
<p class="MsoNormal">The ‘in_port=<OVS-Port-ID>’ check is used to identify packets coming in from the port associated with the VM’s vNIC. This allows us to capture all packets flowing out of the vNIC under question. No problems here.<o:p></o:p></p>
<p class="MsoNormal">Ingress Side (flowing into the vNIC):<o:p></o:p></p>
<p class="MsoNormal">The ‘dl_vlan=<Port-VLAN-tag>, dl_dst=<Port-MAC-Address>’ check is used to identify packets destined for the port associated with the VM’s vNIC.
<o:p></o:p></p>
<p class="MsoNormal">The VLAN tag check was put in place because Neutron port MAC addresses are required to be unique only within a (virtual) network. Now each instance port in br-int gets tagged with a VLAN id that is the same for all ports on that host, which
are associated with a particular (virtual) network. Ports belonging to different (virtual) networks therefore get tagged with different VLAN ids.
<o:p></o:p></p>
<p class="MsoNormal">The combination of “VLAN tag + destination MAC address” was felt to be a sufficient check to ensure that we are truly dealing with traffic heading out of the port being monitored.<o:p></o:p></p>
<p class="MsoNormal"><u>Root Cause of the problem:<o:p></o:p></u></p>
<p class="MsoNormal">The br-int bridge operates using the legacy (or normal) mode of operation. We have observed that under these circumstances, OVS does not add VLAN tags to packets, as long as they are flowing within the bridge. Packets escaping from br-int
into br-tun (via the patch ports), however, get tagged as expected when they arrive in br-tun.
<o:p></o:p></p>
<p class="MsoNormal">Due to this behavior, our ingress side flow (described above) fails to capture packets flowing into a VM’s vNIC, when it is originating from another port on the same host and on the same virtual network.<o:p></o:p></p>
<p class="MsoNormal"><u>Possible Solutions:<o:p></o:p></u></p>
<p class="MsoNormal">OVS documentation (refer: Open vSwitch FAQ) seems to recommend not using the normal operating mode, but handling VLANs explicitly, if flows related to VLAN ids are going to be used. As mentioned above, to ensure that we correctly handle
the Neutron requirement that port MAC addresses are unique only within a virtual network, we need to add VLAN related checks to the TaaS flows. However, br-int today relies on the normal operation mode for most of its work.<o:p></o:p></p>
<p class="MsoNormal">This has left us now in a quandary situation. Here is a proposal to move us forward:<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>As a temporary fix, we can convert the TaaS ingress side flow in br-int from
<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">dl_vlan=<Port-VLAN-tag>, dl_dst=<Port-MAC-Address><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">to<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in"> dl_dst=<Port-MAC-Address><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">This will obviously mean that we no longer support the notion that Neutron port MAC addresses are unique only within a virtual network. However, given that the probability of two port MAC addresses on a host being
the same is low, this should suffice for the short term.<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[We will also disable the flows used to capture broadcast/multicast traffic flowing into a VM’s vNIC]<o:p></o:p></p>
<p class="MsoListParagraphCxSpFirst" style="text-indent:-.25in;mso-list:l0 level1 lfo1">
<![if !supportLists]><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Implement VLAN handling explicitly in br-int and thereby move away from the normal operating mode. This will be a core Neutron change and not limited to just TaaS. However, I have the feeling that there will be other projects (outside
of TaaS) that will sooner or later have a need to detect traffic flowing into a vNIC and they will also run into the same issue we are presently facing.<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle"><o:p> </o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="text-indent:-.25in;mso-list:l0 level1 lfo1">
<![if !supportLists]><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Go back to the (current) TaaS ingress side flow (dl_vlan=<Port-VLAN-tag>, dl_dst=<Port-MAC-Address>) when (2) is implemented. We will then be able to once again support the Neutron requirement that port MAC addresses need be unique
only within a (virtual) network.<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle"><o:p> </o:p></p>
<p class="MsoListParagraphCxSpMiddle">[We will re-enable the flows used to capture broadcast/multicast traffic flowing into a VM’s vNIC]<o:p></o:p></p>
<p class="MsoListParagraphCxSpLast"><o:p> </o:p></p>
<p class="MsoNormal">Thoughts /comments are welcome.<o:p></o:p></p>
<p class="MsoNormal">Thanks and regards,<o:p></o:p></p>
<p class="MsoNormal">Anil<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>