<div dir="ltr"><div> I don't think it's enough for a large scale cloud.</div><div><br></div><div> When the neutron server is not available and the flow rules are gone,</div><div> we need the backup to restore the flow rules.</div><div><br></div><div> We have more than a thousand physical servers in our production environment.</div><div> Rare events will occur where combined failures or unanticipated failures</div><div> require human interaction. For example, a cron job accidentlly killed the</div><div> OvS service(flows will be gone) when one of RabbitMQ, MySQL and neutron</div><div> server is down/unavailable.</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Feb 22, 2016 at 5:44 PM, Ihar Hrachyshka <span dir="ltr"><<a href="mailto:ihrachys@redhat.com" target="_blank">ihrachys@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">Jian Wen <<a href="mailto:wenjianhn@gmail.com" target="_blank">wenjianhn@gmail.com</a>> wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello,<br>
<br>
If we restart OvS/ovs-agent when one or more of Neutron, MySQL and<br>
RabbitMQ is not available, the flow rules in OvS will be gone. If<br>
Neutron/MySQL/RabbitMQ doesn't become available in time, the VMs<br>
will lose their network connections. It's not easy for an<br>
operations engineer to manually restore the flow rules. An<br>
operations engineer working under pressure at 2 a.m. will make<br>
mistakes.<br>
<br>
We can backup the ports info to a local file. In case of emergency<br>
the ovs-agent can use it to restore the flow rules. What do you<br>
think of this feature?<br>
<br>
Related bugs:<br>
Restarting neutron openvswitch agent causes network hiccup by throwing away all flows<br>
<a href="https://bugs.launchpad.net/neutron/+bug/1383674" rel="noreferrer" target="_blank">https://bugs.launchpad.net/neutron/+bug/1383674</a><br>
<br>
Restarting OVS agent drops VMs traffic when using VLAN provider bridges<br>
<a href="https://bugs.launchpad.net/neutron/+bug/1514056" rel="noreferrer" target="_blank">https://bugs.launchpad.net/neutron/+bug/1514056</a><br>
<br>
After restarting an ovs agent, it still drops useful flows if the neutron server is busy/down<br>
<a href="https://bugs.launchpad.net/neutron/+bug/1515075" rel="noreferrer" target="_blank">https://bugs.launchpad.net/neutron/+bug/1515075</a><br>
<br>
Ovs agent loses OpenFlow rules if OVS gets restarted while Neutron is disconnected from SQL<br>
<a href="https://bugs.launchpad.net/neutron/+bug/1531210" rel="noreferrer" target="_blank">https://bugs.launchpad.net/neutron/+bug/1531210</a><br>
<br>
</blockquote>
<br></span>
Most of those bugs are fixed (at least for stable/liberty+). Isn’t it enough to avoid data plane reset when the agent fails to fetch new port data from its controller? Why do we need another mechanism here?<br>
<br>
Ihar<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">Best,<br><br>Jian<br></div>
</div>