<div dir="ltr">Good to see the <span style="font-size:12.8px">priority of the bug has been upgraded.</span><div><span style="font-size:12.8px"><br></span><div><span style="font-size:12.8px">When we hit the bug, everyone was under huge pressure.</span></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 12, 2016 at 11:01 PM, Clayton O'Neill <span dir="ltr"><<a href="mailto:clayton@oneill.net" target="_blank">clayton@oneill.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I’ve tried it with both a blank value and the specific value. It<br>
doesn’t appear to make a difference.<br>
<br>
In other news, Assaf Muller has upgraded the priority of the bug from<br>
low to high.<br>
<div class="HOEnZb"><div class="h5"><br>
On Fri, Feb 12, 2016 at 9:27 AM, Matt Kassawara <<a href="mailto:mkassawara@gmail.com">mkassawara@gmail.com</a>> wrote:<br>
> Out of curiosity, what do you have for the "external_network_bridge" option<br>
> in the L3 agent config?<br>
><br>
> On Wed, Feb 10, 2016 at 2:42 PM, Bajin, Joseph <<a href="mailto:jbajin@verisign.com">jbajin@verisign.com</a>> wrote:<br>
>><br>
>> Clayton,<br>
>><br>
>> This is really good information.<br>
>><br>
>> I’m wondering how we can help support you and get the necessary dev<br>
>> support to get this resolved sooner than later. I totally agree with you<br>
>> that this should be backported to at least Liberty.<br>
>><br>
>> Please let me know how I and other can help!<br>
>><br>
>> —Joe<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On 2/10/16, 8:55 AM, "Clayton O'Neill" <<a href="mailto:clayton@oneill.net">clayton@oneill.net</a>> wrote:<br>
>><br>
>> >Summary: Liberty OVS agent restarts are better, but still need work.<br>
>> >See: <a href="https://bugs.launchpad.net/neutron/+bug/1514056" rel="noreferrer" target="_blank">https://bugs.launchpad.net/neutron/+bug/1514056</a><br>
>> ><br>
>> >As many of you know, Liberty has a fix for OVS agent restarts such<br>
>> >that it doesn’t dump all flows when starting, resulting in a loss of<br>
>> >traffic. Unfortunately, Liberty neutron still has issues with OVS<br>
>> >agent restarts. The fix that went into Liberty prevents it from<br>
>> >dropping flows on the br-tun and br-int bridges and that helps<br>
>> >greatly, but the br-ex bridge still has it’s flows cleared on startup.<br>
>> ><br>
>> >You may be thinking: Wait, br-ex only has like 3 flows on it, how can<br>
>> >that be a problem? The issue appears to be that the br-ex flows are<br>
>> >cleared early and not setup again until late in the process. This<br>
>> >means that routers on the node where OVS agent is lose network<br>
>> >connectivity for the majority of the restart time.<br>
>> ><br>
>> >I did some testing with this yesterday, comparing a few scenarios with<br>
>> >100 FIPS, 100 instances and various scenarios for routers. You can<br>
>> >find the the complete data here:<br>
>><br>
>> > ><a href="https://docs.google.com/spreadsheets/d/1ZGra_MszBlL0fNsFqd4nOvh1PsgWu58-GxEeh1m1BPw/edit?usp=sharing" rel="noreferrer" target="_blank">https://docs.google.com/spreadsheets/d/1ZGra_MszBlL0fNsFqd4nOvh1PsgWu58-GxEeh1m1BPw/edit?usp=sharing</a><br>
>> ><br>
>> >The summary looks like this:<br>
>> >100 routers, 100 networks, 100 floating ips, 100 instances, single node<br>
>> > test:<br>
>> >Kilo average outage time: 47 seconds<br>
>> >Liberty average outage time: 37 seconds<br>
>> ><br>
>> >1 router, 1 network, 100 floating ips, 100 instances, single node test:<br>
>> >Kilo average outage time: 46 seconds<br>
>> >Liberty average outage time: 13 seconds<br>
>> ><br>
>> >1 router, 1 network, 100 floating its, 100 instances, router on a<br>
>> >separate node, all instances on a single node, OVS restart on compute<br>
>> >node:<br>
>> >Kilo average outage time: 25 seconds<br>
>> >Liberty average outage time: 0 to 1 seconds<br>
>> ><br>
>> >I did my testing using 1 second pings using fping to all of the<br>
>> >floating IPs. With the last test, it frequently lost no packets, and<br>
>> >as a result I was not really able to test the scenario other than to<br>
>> >qualify it as good.<br>
>> ><br>
>> >This is a huge operational issue for us and I suspect for many of the<br>
>> >rest of you using OVS. I’d encourage everyone that is using OVS to<br>
>> >register interest in having this fixed in the LP bug<br>
>> >(<a href="https://bugs.launchpad.net/neutron/+bug/1514056" rel="noreferrer" target="_blank">https://bugs.launchpad.net/neutron/+bug/1514056</a>). Right now this bug<br>
>> >as marked as low priority.<br>
>> ><br>
>> >_______________________________________________<br>
>> >OpenStack-operators mailing list<br>
>> ><a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
>> ><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
>><br>
>> _______________________________________________<br>
>> OpenStack-operators mailing list<br>
>> <a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
>><br>
><br>
<br>
_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">Best,<br><br>Jian<br></div>
</div>