<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
Hi Kevin</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
The information you asked for</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
For 1 compute node with 45 Vms here is the number of connection tracking entries getting deleted</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div>cat conntrack.file | wc -l</div>
<div> 38528</div>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
The file with output is 14MB so ill email it to Ian and he can share it if needed</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
Security group rules</div>
<div>
<div><font face="Calibri,sans-serif"><span class="Apple-tab-span" style="white-space: pre;"></span>Direction<span class="Apple-tab-span" style="white-space: pre;">
</span>Ether Type<span class="Apple-tab-span" style="white-space: pre;"> </span>IP Protocol<span class="Apple-tab-span" style="white-space: pre;">
</span>Port Range<span class="Apple-tab-span" style="white-space: pre;"> </span>Remote IP Prefix<span class="Apple-tab-span" style="white-space: pre;">
</span>Remote Security Group<span class="Apple-tab-span" style="white-space: pre;">
</span>Actions</font></div>
<div><font face="Calibri,sans-serif">Egress<span class="Apple-tab-span" style="white-space:pre">
</span>IPv4<span class="Apple-tab-span" style="white-space:pre"> </span>Any<span class="Apple-tab-span" style="white-space:pre">
</span>Any<span class="Apple-tab-span" style="white-space:pre"> </span>0.0.0.0/0<span class="Apple-tab-span" style="white-space:pre">
</span></font></div>
<div><font face="Calibri,sans-serif">Ingress<span class="Apple-tab-span" style="white-space:pre">
</span>IPv6<span class="Apple-tab-span" style="white-space:pre"> </span>Any<span class="Apple-tab-span" style="white-space:pre">
</span>Any<span class="Apple-tab-span" style="white-space:pre"> </span>-<span class="Apple-tab-span" style="white-space:pre">
</span>default<span class="Apple-tab-span" style="white-space:pre"> </span></font></div>
<div><font face="Calibri,sans-serif">Egress<span class="Apple-tab-span" style="white-space:pre">
</span>IPv6<span class="Apple-tab-span" style="white-space:pre"> </span>Any<span class="Apple-tab-span" style="white-space:pre">
</span>Any<span class="Apple-tab-span" style="white-space:pre"> </span>::/0<span class="Apple-tab-span" style="white-space:pre">
</span>-<span class="Apple-tab-span" style="white-space:pre"> </span></font></div>
<div><font face="Calibri,sans-serif">Ingress<span class="Apple-tab-span" style="white-space: pre;">
</span>IPv4<span class="Apple-tab-span" style="white-space: pre;"> </span>Any<span class="Apple-tab-span" style="white-space: pre;">
</span>Any<span class="Apple-tab-span" style="white-space: pre;"> </span>-<span class="Apple-tab-span" style="white-space: pre;">
</span></font></div>
</div>
<div><font face="Calibri,sans-serif"><br>
</font></div>
<div><font face="Calibri,sans-serif">Please let me know if u need the dump of conntrack entries if so I can email it to email address of your choice</font></div>
<div><font face="Calibri,sans-serif"><br>
</font></div>
<div><font face="Calibri,sans-serif"><br>
</font></div>
<div><font face="Calibri,sans-serif">Ajay</font></div>
<div><font face="Calibri,sans-serif"><br>
</font></div>
<div><font face="Calibri,sans-serif"><br>
</font></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div id="MAC_OUTLOOK_SIGNATURE"></div>
</div>
</div>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<span id="OLK_SRC_BODY_SECTION" style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div style="font-family:Calibri; font-size:12pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Ajay Kalambur <<a href="mailto:akalambu@cisco.com">akalambu@cisco.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Monday, September 11, 2017 at 10:02 AM<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [neutron]OVS connection tracking cleanup<br>
</div>
<div><br>
</div>
<span style="mso-bookmark:_MailOriginalBody">
<div>
<div dir="auto">
<div>Hi Kevin</div>
<div id="AppleMailSignature">Thanks for your response it was about 50 vms </div>
<div id="AppleMailSignature">Ajay<br>
<br>
<br>
</div>
<div><br>
On Sep 11, 2017, at 9:49 AM, Kevin Benton <<a href="mailto:kevin@benton.pub">kevin@benton.pub</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">The biggest improvement will be switching to native netlink calls: <a href="https://review.openstack.org/#/c/470912/">https://review.openstack.org/#/c/470912/</a>
<div><br>
</div>
<div>How many VMs were on a single compute node?</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Sep 11, 2017 at 9:15 AM, Ajay Kalambur (akalambu)
<span dir="ltr"><<a href="mailto:akalambu@cisco.com" target="_blank">akalambu@cisco.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">Hi</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">I am performing a scale test and I see that after creating 500 Vms with ping traffic between them it took almost 1 hr for the connection tracking </div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">To clean up and ovs agent was busy doing this and unable to service any new port bind requests on some computes for almost an hr </div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">It took that long for conntrack clean up to complete</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px"><br>
</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px"><br>
</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">I see the following bug</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px"><a href="https://bugs.launchpad.net/neutron/+bug/1513765" target="_blank">https://bugs.launchpad.net/<wbr>neutron/+bug/1513765</a></div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px"><br>
</div>
<div>And I also have the fix below</div>
<div><a href="https://git.openstack.org/cgit/openstack/neutron/commit/?id=d7aeb8dd4b1d122e17eef8687192cd122b79fd6e" target="_blank">https://git.openstack.org/<wbr>cgit/openstack/neutron/commit/<wbr>?id=<wbr>d7aeb8dd4b1d122e17eef8687192cd<wbr>122b79fd6e</a></div>
<div><br>
</div>
<div><br>
</div>
<div>Still see really long times for conntrack cleanup</div>
<div><br>
</div>
<div>What is the solution to this problem in scale scenarios?</div>
<span class="HOEnZb"><font color="#888888">
<div>Ajay</div>
<div><br>
</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">
<div id="m_-6662988751102660720MAC_OUTLOOK_SIGNATURE"></div>
</div>
</font></span></div>
<br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<blockquote type="cite">
<div><span>__________________________________________________________________________</span><br>
<span>OpenStack Development Mailing List (not for usage questions)</span><br>
<span>Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org">OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe</span><br>
<span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span><br>
</div>
</blockquote>
</div>
</div>
</span></span>
</body>
</html>