<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.tlid-translation
{mso-style-name:tlid-translation;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">I seem to recall checking this when the issue was first discovered, and OVN did not appear to implement the same flow rules that resulted in the issue. I don’t have a live environment to test with, though.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">James Denton<o:p></o:p></p>
<p class="MsoNormal">Network Engineer<o:p></o:p></p>
<p class="MsoNormal">Rackspace Private Cloud<o:p></o:p></p>
</div>
<p class="MsoNormal">james.denton@rackspace.com<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Volodymyr Litovka <doka.ua@gmx.com><br>
<b>Date: </b>Friday, November 15, 2019 at 3:55 AM<br>
<b>To: </b>James Denton <james.denton@rackspace.com>, "openstack-discuss@lists.openstack.org" <openstack-discuss@lists.openstack.org>, Slawek Kaplonski <skaplons@redhat.com><br>
<b>Cc: </b>"doka.ua@gmx.com" <doka.ua@gmx.com><br>
<b>Subject: </b>Re: [Neutron] OVS forwarding issues<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="MsoNormal" style="line-height:12.0pt;background:#FFEB9C"><b><span style="font-size:10.0pt;color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt;color:black"> This message originated externally, please use caution when clicking on links or
opening attachments!<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi colleagues,<br>
<br>
thanks for the pointing on this. Can anybody _assume_ whether this bug affects also ML2/OVN implementation of networking?<br>
<br>
I was looking into OVN sometimes ago, but due to lack of resources skipped this research, now I think it makes sense to return back to this question.<br>
<br>
Thank you.<br>
<br>
<o:p></o:p></p>
<div>
<p class="MsoNormal">On 11.11.2019 19:38, James Denton wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">This is a known issue with the openvswitch firewall[1].<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">> firewall_driver = openvswitch<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">I recommend running iptables_hybrid until that is resolved.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">[1] <a href="https://bugs.launchpad.net/neutron/+bug/1732067">
https://bugs.launchpad.net/neutron/+bug/1732067</a><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<p class="MsoNormal">James Denton<o:p></o:p></p>
<p class="MsoNormal">Network Engineer<o:p></o:p></p>
<p class="MsoNormal">Rackspace Private Cloud<o:p></o:p></p>
</div>
<p class="MsoNormal"><a href="mailto:james.denton@rackspace.com">james.denton@rackspace.com</a><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Volodymyr Litovka
<a href="mailto:doka.ua@gmx.com"><doka.ua@gmx.com></a><br>
<b>Date: </b>Monday, November 11, 2019 at 12:10 PM<br>
<b>To: </b><a href="mailto:openstack-discuss@lists.openstack.org">"openstack-discuss@lists.openstack.org"</a>
<a href="mailto:openstack-discuss@lists.openstack.org"><openstack-discuss@lists.openstack.org></a><br>
<b>Cc: </b><a href="mailto:doka.ua@gmx.com">"doka.ua@gmx.com"</a> <a href="mailto:doka.ua@gmx.com">
<doka.ua@gmx.com></a><br>
<b>Subject: </b>[Neutron] OVS forwarding issues</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="MsoNormal" style="line-height:12.0pt;background:#FFEB9C"><b><span style="font-size:10.0pt;color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt;color:black"> This message originated externally, please use caution when clicking on links or
opening attachments!</span><o:p></o:p></p>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<p class="MsoNormal">Dear colleagues,<br>
<br>
just faced an issue with Openvswitch, which looks strange for me. The problem is that any particular VM receives a lot of packets, which are unicasted:<br>
- from other VMs which reside on the same host (let's name them "local VMs")<br>
- to other VMs which reside on other hosts (let's name them "remote VMs")<br>
<br>
Long output from "ovs-ofctl dump-flows br-int" which, as far as I can narrow, ends there:<br>
<br>
# ovs-ofctl dump-flows br-int |grep " table=94," |egrep "n_packets=[123456789]"<br>
cookie=0xaf6b1435fe826bdf, duration=2952350.695s, table=94, n_packets=291494723, n_bytes=40582103074, idle_age=0, hard_age=65534, priority=1 actions=NORMAL<br>
<br>
coming to normal processing (classic MAC learning). Looking into br-int MAC-table (ovs-appctl fdb/show br-int) shows, that there are really no MAC addresses of remote VMs and br-int behaves in the right way, flooding unknown unicast to all ports in this L2
segment.<br>
<br>
Of course, there is br-tun which connected over vxlan to all other hosts and to br-int:<br>
<br>
Bridge br-tun<br>
Controller "tcp:127.0.0.1:6633"<br>
is_connected: true<br>
fail_mode: secure<br>
Port "vxlan-0a960008"<br>
Interface "vxlan-0a960008"<br>
type: vxlan<br>
options: {df_default="true", in_key=flow, local_ip="10.150.0.5", out_key=flow, remote_ip="10.150.0.8"}<br>
[ ... ]<br>
Port br-tun<br>
Interface br-tun<br>
type: internal<br>
Port patch-int<br>
Interface patch-int<br>
type: patch<br>
options: {peer=patch-tun}<br>
<br>
but MAC table on br-tun is empty as well:<br>
<br>
# ovs-appctl fdb/show br-tun<br>
port VLAN MAC Age<br>
#<br>
<br>
Finally, packets get to destination, while being copied to all ports on source host, which is
<span class="tlid-translation"><span lang="EN">serious security issue.</span></span><br>
<br>
<span class="tlid-translation"><span lang="EN">I do not think so conceived by design</span></span>, I rather think we missed something in configuration. Can anybody point me where we're wrong and help with this issue?<br>
<br>
We're using Openstack Rocky and OVS 2.10.0 under Ubuntu 16.04. Network configuration is:<br>
<br>
@controller:<br>
# cat /etc/neutron/plugins/ml2/ml2_conf.ini |egrep -v "^$|^#"<br>
[DEFAULT]<br>
verbose = true<br>
[ml2]<br>
type_drivers = flat,vxlan<br>
tenant_network_types = vxlan<br>
mechanism_drivers = l2population,openvswitch<br>
extension_drivers = port_security,qos,dns_domain_ports<br>
[ml2_type_flat]<br>
flat_networks = provider<br>
[ml2_type_geneve]<br>
[ml2_type_gre]<br>
[ml2_type_vlan]<br>
[ml2_type_vxlan]<br>
vni_ranges = 400:400000<br>
[securitygroup]<br>
firewall_driver = openvswitch<br>
enable_security_group = true<br>
enable_ipset = true<br>
<br>
@agent:<br>
# cat /etc/neutron/plugins/ml2/openvswitch_agent.ini |egrep -v "^$|^#"<br>
[DEFAULT]<br>
verbose = true<br>
[agent]<br>
tunnel_types = vxlan<br>
l2_population = true<br>
arp_responder = true<br>
extensions = qos<br>
[ovs]<br>
local_ip = 10.150.0.5<br>
bridge_mappings = provider:br-ex<br>
[securitygroup]<br>
firewall_driver = openvswitch<br>
enable_security_group = true<br>
enable_ipset = true<br>
[xenapi]<br>
<br>
Thank you.<br>
<br>
<br>
<br>
<o:p></o:p></p>
<pre>--<o:p></o:p></pre>
<pre>Volodymyr Litovka<o:p></o:p></pre>
<pre> "Vision without Execution is Hallucination." -- Thomas Edison<o:p></o:p></pre>
</div>
</blockquote>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<pre>--<o:p></o:p></pre>
<pre>Volodymyr Litovka<o:p></o:p></pre>
<pre> "Vision without Execution is Hallucination." -- Thomas Edison<o:p></o:p></pre>
</div>
</div>
</body>
</html>