<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On 13 February 2017 at 23:23, Kosnik, Lubosz <span dir="ltr"><<a href="mailto:lubosz.kosnik@intel.com" target="_blank">lubosz.kosnik@intel.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
So from my perspective I can tell that problem is completely in architecture and even without something outside of Neutron we cannot solve that.
<div>Two releases ago I started to work on hardening that feature but all my ideas was killed by Armando and Assaf. The decided that adding outside dependency will open the doors for a new bugs from dependencies into Neutron [1].</div></div></blockquote><div><br></div><div>I am pretty sure it wasn't our intentions to 'kill' your ideas, but otherwise set you on the right path for fixing the bug. I still believe that a complete and robust L3 HA solution cannot be built solely with Neutron alone, and that's what I was trying to say with the comment referenced below.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">
<div><br>
</div>
<div>You need to know that there are two outstanding bugs in this feature. There is a internal and outside connectivity split brain. [2] this patch made by me is “fixing” part of the problem. It allows you specify additional tests to verify connectivity
from router to GW.</div>
<div>Also there is a problem with connectivity between network nodes. It’s more problematic and like you said it’s unsolvable in my opinion without using external mechanism.</div>
<div><br>
</div>
<div>If there will be any need to help with anything I would love to help with sharing my knowledge about this feature and what exactly is not working. If anyone needs any help with anything about this please ping me on email or IRC.</div>
<div><br>
</div>
<div>[1] <a href="https://bugs.launchpad.net/neutron/+bug/1375625/comments/31" target="_blank">https://bugs.launchpad.<wbr>net/neutron/+bug/1375625/<wbr>comments/31</a></div>
<div>[2] <a href="https://review.openstack.org/#/c/273546/" target="_blank">https://review.openstack.<wbr>org/#/c/273546/</a></div><span class="HOEnZb"><font color="#888888">
</font></span><div><span class="HOEnZb"><font color="#888888"><br>
<div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div>
<div>Lubosz</div>
</div>
<div><br>
</div>
</div>
</div></font></span><div><div class="h5">
<div>
<blockquote type="cite">
<div>On Feb 13, 2017, at 4:10 AM, Anna Taraday <<a href="mailto:akamyshnikova@mirantis.com" target="_blank">akamyshnikova@mirantis.com</a>> wrote:</div>
<br class="m_-4921772524764041929Apple-interchange-newline">
<div>
<div dir="ltr">To avoid dependency of data plane on control plane it is possible to deploy a separate key-value storage cluster on data plane side, using the same network nodes.
<div>I'm proposing to make some changes to enable experimentation in this field, we are yet to come up with any other concrete solution.</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Mon, Feb 13, 2017 at 2:01 PM <<a href="mailto:cristi.calin@orange.com" target="_blank">cristi.calin@orange.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple" class="m_-4921772524764041929gmail_msg">
<div class="m_-4921772524764041929m_6440967129772186155WordSection1 m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg">Hi,<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg">We also operate using Juno with the VRRP HA implementation and at had to patch through several bugs before getting to the Mitaka
release.<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg">An pluggable, drop-in alternative would be highly appreciated. However our experience has been that the decoupling of VRRP from
the control plane is actually a benefit as when the control plane is down the traffic is not affected.<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg">In a solution where the L3 HA implementation becomes tied to the availability of the control plane (etcd cluster or any other KV
store) then an operator would have to account for extra failure scenarios for the KV store which would affect multiple routers than the outage of a single L3 node which is the case we usually have to account now.<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg">Just my $.02<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg">Cristian<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" class="m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></span></p>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><b class="m_-4921772524764041929gmail_msg"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" class="m_-4921772524764041929gmail_msg">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" class="m_-4921772524764041929gmail_msg"> Anna Taraday [mailto:<a href="mailto:akamyshnikova@mirantis.com" class="m_-4921772524764041929gmail_msg" target="_blank">akamyshnikova@<wbr>mirantis.com</a>]
<br class="m_-4921772524764041929gmail_msg">
<b class="m_-4921772524764041929gmail_msg">Sent:</b> Monday, February 13, 2017 11:45 AM<br class="m_-4921772524764041929gmail_msg">
<b class="m_-4921772524764041929gmail_msg">To:</b> OpenStack Development Mailing List (not for usage questions)<br class="m_-4921772524764041929gmail_msg">
<b class="m_-4921772524764041929gmail_msg">Subject:</b> Re: [openstack-dev] [Neutron] Alternative approaches for L3 HA<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></span></p>
</div>
</div>
<div lang="EN-US" link="blue" vlink="purple" class="m_-4921772524764041929gmail_msg">
<div class="m_-4921772524764041929m_6440967129772186155WordSection1 m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></p>
<div class="m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg">In etcd for each HA router we can store key which will identify which agent is active. L3 agents will "watch" this key.<br class="m_-4921772524764041929gmail_msg">
All these tools have leader election mechanism which can be used to get agent which is active for current HA router. <u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></p>
</div>
<p class="MsoNormal m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></p>
<div class="m_-4921772524764041929gmail_msg">
<div class="m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg">On Mon, Feb 13, 2017 at 7:02 AM zhi <<a href="mailto:changzhi1990@gmail.com" class="m_-4921772524764041929gmail_msg" target="_blank">changzhi1990@gmail.com</a>> wrote:<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in" class="m_-4921772524764041929gmail_msg">
<div class="m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg">Hi, we are using L3 HA in our production environment now. Router instances communicate to each other by VRRP protocol. In my opinion, although VRRP is a control plane thing, but the real VRRP traffic is using data plane nic so
that router namespaces can not talk to each other sometimes when the data plan is busy. If we were used etcd (or other), does every router instance register one "id" in etcd ?<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></p>
<div class="m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></p>
</div>
<div class="m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg"><u class="m_-4921772524764041929gmail_msg"></u> <u class="m_-4921772524764041929gmail_msg"></u></p>
</div>
<div class="m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg">Thanks <u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></p>
</div>
<div class="m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg">Zhi Chang <u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></p>
</div>
</div>
<p class="MsoNormal m_-4921772524764041929gmail_msg">______________________________<wbr>______________________________<wbr>______________<br class="m_-4921772524764041929gmail_msg">
OpenStack Development Mailing List (not for usage questions)<br class="m_-4921772524764041929gmail_msg">
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe" class="m_-4921772524764041929gmail_msg" target="_blank">
OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br class="m_-4921772524764041929gmail_msg">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" class="m_-4921772524764041929gmail_msg" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></p>
</blockquote>
</div>
<div class="m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg">-- <u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></p>
</div>
<div class="m_-4921772524764041929gmail_msg">
<div class="m_-4921772524764041929gmail_msg">
<p class="MsoNormal m_-4921772524764041929gmail_msg">Regards,<br class="m_-4921772524764041929gmail_msg">
Ann Taraday<u class="m_-4921772524764041929gmail_msg"></u><u class="m_-4921772524764041929gmail_msg"></u></p>
</div>
</div>
</div>
</div>
<div lang="EN-US" link="blue" vlink="purple" class="m_-4921772524764041929gmail_msg">
<div class="m_-4921772524764041929m_6440967129772186155WordSection1 m_-4921772524764041929gmail_msg"></div>
<pre class="m_-4921772524764041929gmail_msg">______________________________<wbr>______________________________<wbr>______________________________<wbr>______________________________<wbr>_
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
</pre>
</div>
______________________________<wbr>______________________________<wbr>______________<br class="m_-4921772524764041929gmail_msg">
OpenStack Development Mailing List (not for usage questions)<br class="m_-4921772524764041929gmail_msg">
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe" rel="noreferrer" class="m_-4921772524764041929gmail_msg" target="_blank">
OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br class="m_-4921772524764041929gmail_msg">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" class="m_-4921772524764041929gmail_msg" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br class="m_-4921772524764041929gmail_msg">
</blockquote>
</div>
<div dir="ltr">-- <br>
</div>
<div data-smartmail="gmail_signature">
<div dir="ltr">Regards,<br>
Ann Taraday</div>
</div>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" target="_blank">
OpenStack-dev-request@lists.<wbr>openstack.org</a>?subject:<wbr>unsubscribe<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
</div>
</blockquote>
</div>
<br>
</div></div></div>
</div>
<br>______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
<br></blockquote></div><br></div></div>