<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">OVS 2.0.1 is still in plans. <br>
<br>
But I found the source of networking delays!<br>
<br>
If someone touch ovs configuration (f.e. add iface to bridge) it
save state to internal database and restore it on next boot.<br>
<br>
But if br-int has some tun/tap interfaces, they gonna be saved
too. With all gre-rules. And restored on next boot. Before nova
start any instances. That cause some huge delay and sometime
irresponsiveness of floatingips, and even DHCP gonna ask for lease
few times, or even fail completely to get lease due timeout.<br>
<br>
I don't know proper solution so far, but for now it just rule 'how
to change OVS config':<br>
<br>
* schedule downtime<br>
* shutoff/migrate every instance<br>
* disable routers/dhcp agents (for networking node)<br>
* change configuration<br>
* clean bridges of all strange interfaces<br>
* move instances back or start them<br>
<br>
Situation is bit simpler for netboot nodes, where is no local
'configuration database' on the node.<br>
<br>
<br>
On 24.01.2014 00:51, Jacob Godin wrote:<br>
</div>
<blockquote
cite="mid:CA+WiSK7qtN-OrRQYiWS0zrVg0s39EaLP3CQefXuanR5LBfPS3A@mail.gmail.com"
type="cite">
<div dir="ltr">Hi George,
<div><br>
</div>
<div>Thanks for the detailed response. How has OVS 2 been
working for you so far? </div>
<div><br>
</div>
<div>I'm running the 3.5 kernel from Precise
(generic-lts-quantal), so hopefully the kernel module won't be
an issue. Not sure why that delay would occur after boot..</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Sun, Jan 19, 2014 at 3:45 PM, George
Shuklin <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:george.shuklin@gmail.com" target="_blank">george.shuklin@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Yes and no.<br>
<br>
Yes: I was able to upgrade laboratory cluster from OVS
1.10 to OVS 1.11 and it performs few orders better under
--rand-source DoS attack then OVS 1.10-based installation.<br>
No: there is issues.<br>
<br>
Issue #1:<br>
OVS 1.11 (vanilla version) has datapath (kernel module)
with is not compilable with linux-3.11 (which is default
for ubuntu cloud archive). Because canonical was able to
build OVS 1.10 against 3.11, I think this is possible.
Research pending, but right now I stuck with linux-3.8<br>
<br>
Issue #2:<br>
Delayed recovery after reboot. Because of unknown reason
(research pending) systems under OVS 1.11 behave bit
strange after whole system (all hosts) reboot. There is a
long delay (about 6-10 minutes) before networking restore
after successful booting of every server and instance
start. At first I even thought it is broken (uptime 3
minutes - no dhcp for instances).<br>
<br>
I'll continue to play around ovs 2.0.1 and other questions
with networking, because deploying OVS 1.10 to production
environment is some kind of slow suicide. Any script
kiddie with hping and just 15 Mbit channel will able to
completely shutoff networking node (>90% packet loss),
and just about 5Mbit/s of --rand-source flood is enough to
cripple it (>5% packet loss).
<div>
<div class="h5"><br>
<br>
<div>On 01/18/2014 04:24 PM, Jacob Godin wrote:<br>
</div>
<blockquote type="cite">
<p dir="ltr">Hi George,</p>
<p dir="ltr">To clarify, you were able to upgrade
from 1.10 or install 1.11 fresh without any
issues?</p>
<p dir="ltr">Sent from my mobile device</p>
<div class="gmail_quote">On Jan 17, 2014 4:56 PM,
"George Shuklin" <<a moz-do-not-send="true"
href="mailto:george.shuklin@gmail.com"
target="_blank">george.shuklin@gmail.com</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> For 1.11
I was wrong, it working fine. <br>
<br>
For 2.0.1 something is broken, but I still
can't get where. VMs can ping each other
within host (if configured manually), but
traffic is not getting out br-tun (no GRE, no
DHCP from network node).<br>
<br>
<div>On 01/16/14 18:11, Aaron Rosen wrote:<br>
</div>
<blockquote type="cite">
<p dir="ltr">Hi,</p>
<p dir="ltr">Can you give more details on
how it breaks? Did you restart the agents
so it reprograms the flows back down? </p>
<div class="gmail_quote">On Jan 16, 2014
2:06 AM, "George Shuklin" <<a
moz-do-not-send="true"
href="mailto:george.shuklin@gmail.com"
target="_blank">george.shuklin@gmail.com</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex"> Good day.<br>
<br>
Did anyone successfully combine havanna
and OVS > 1.10? OVS 1.10 is really
suck under specific types of load (was
fixed in OVS 1.11 and later). But plain
upgrade of OVS breaks neutron (under
research).<br>
<br>
Did anyone walk that path?<br>
<br>
Thanks.<br>
<br>
_______________________________________________<br>
OpenStack-operators mailing list<br>
<a moz-do-not-send="true"
href="mailto:OpenStack-operators@lists.openstack.org"
target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
</blockquote>
</div>
</blockquote>
<br>
</div>
<br>
_______________________________________________<br>
OpenStack-operators mailing list<br>
<a moz-do-not-send="true"
href="mailto:OpenStack-operators@lists.openstack.org"
target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br>
</blockquote>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>