<div dir="ltr"><div>Hello Roberto:</div><div><br></div><div>I think you have provided a very detailed explanation of the issue but you still didn't present any possible solution. I would ask you at least for a proposal and for specific details of what is failing in the OVN sync tool.</div><div><br></div><div>For example, if you add a TS between two clouds, you'll need to create (1) routers (NAT registers), (2) router ports (LRP) and (3) some static routes (LRSR). All these elements are monitored by the DB sync tool and will fail because the counterparts in the Neutron DB don't exist. I guess that you have some script or tool or at least a document to manually add these resources; sharing it could help to find a solution.</div><div><br></div><div>If these elements are manually created during the deployment phase, you can also control the information provided. I'm thinking about the "external_ids" field; we can push some defined constant that will make these registers transparent to the OVN DB sync tool.</div><div><br></div><div>Please check this idea and if it is feasible. In that case, please add a topic to the Neutron drivers meeting agenda [1] to discuss it.</div><div><br></div><div>Regards.<br></div><div><br></div><div>[1]<a href="https://wiki.openstack.org/wiki/Meetings/NeutronDrivers">https://wiki.openstack.org/wiki/Meetings/NeutronDrivers</a></div><div><br></div><div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 12, 2023 at 2:12 PM Roberto Bartzen Acosta <<a href="mailto:roberto.acosta@luizalabs.com">roberto.acosta@luizalabs.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Em qua., 12 de jul. de 2023 às 09:05, Roberto Bartzen Acosta <<a href="mailto:roberto.acosta@luizalabs.com" target="_blank">roberto.acosta@luizalabs.com</a>> escreveu:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Rodolfo,<div><br></div><div>Thanks for the feedback.</div><div><br></div><div>I liked the idea of having a different network type only to hold the Transit Switches and thus the LRP but it depends on some factors like multi-tenancy. </div><div>From the OVN perspective, the TS is global and will only be linked to a tenant when it is plugged into the tenant router port. My concern here is that if Neutron manages the TS, it will need to assume the dynamic characteristics of this TS function, ovn-ic creates and removes the TS in NB_Global, in addition to plugging and removing ports in this switch according to the network topology. Additionally, Neutron would still need to handle the learned remote routes (which are listed as static routes from the tenant router perspective).</div></div></blockquote><div>sorry: <strike>NB_Global</strike> -> Northbound Database.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>This is an open discussion, Felix can add ideas here.<br></div><div><br></div><div>In general, it seems to me that we have no alternatives for this type of solution other than OVN-IC (note that ovn-bgp-agent does not learn remote routes at the SDN level / OVN). </div><div>OpenStack seems to be designed to run like a "bubble" and this traffic from one VM to another always needs to be routed at the FIP level and external routing layers.<br></div><div><br></div><div>Regards,</div><div>Roberto</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Em qua., 12 de jul. de 2023 às 05:11, Rodolfo Alonso Hernandez <<a href="mailto:ralonsoh@redhat.com" target="_blank">ralonsoh@redhat.com</a>> escreveu:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hello Roberto:</div><div><br></div><div>We talked about this possible RFE during the PTG [1] but I don't remember having any proposal. Actually what I remember (and was written in the etherpad) is that you were going to present an RFE. Can you explain it briefly?<br></div><div></div><div><br></div><div>We also talked about the idea, proposed by Felix in the mailing list, of having a different network type only to hold the Transit Switches and thus the LRP. If you have a proposal, please present it.<br></div><div><br></div><div>Regards.</div><div><br></div><div>[1]<a href="https://etherpad.opendev.org/p/neutron-bobcat-ptg#L506" target="_blank">https://etherpad.opendev.org/p/neutron-bobcat-ptg#L506</a></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jul 11, 2023 at 10:50 PM Roberto Bartzen Acosta <<a href="mailto:roberto.acosta@luizalabs.com" target="_blank">roberto.acosta@luizalabs.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hello Neutron folks,<div><br>Regarding the conversation started in March [1] about the use of OVN interconnect with Neutron, I would like to evolve the discussion in relation to resources allocated by OVN-IC and which are not managed by Neutron. In the March PTG [2], the problem related to the db_sync tool was presented, and a proposed solution in which Neutron does not manage these resources.<br><br>After discussing some architectural designs with Felix/StackIT, it seems to make sense to come up with a proposal to change the mech_driver to validate the external_ids key and not remove resources present in the OVN backend without Neutron "signature".<br><br>The discussion here is more complex than it seems, because architecturally Neutron does not allow us to interconnect workloads in multiple AZs (with different OpenStacks), but this could be a requirement for a high availability cloud solution (Cloud Region). Additionally, this OVN-IC solution allows interconnecting other cloud backends that use OVN, in the case of kubernetes with ovn-kube.<br><br>We tested an OVN interconnect integrated with 3 OpenStack installations and it worked very well !!! considering direct L3 traffic at the router level between different network infrastructures.<br><br>SYNC_REPAIR - problem<br><br>* Static Routes (learned OVN-IC routes)<br>* Router Port -> Transit Switches<br><br>Jul 10 18:34:11 os-infra-1-neutron-server-container-845157ae neutron-server[8632]: 2023-07-10 18:34:11.343 8632 WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [req-8d513732-f932-47b8-bc2c-937958c30f47 - - - - -] Router Port found in OVN but not in Neutron, port_id=rt2-admin-tenant1<br>Jul 10 18:34:11 os-infra-1-neutron-server-container-845157ae neutron-server[8632]: 2023-07-10 18:34:11.343 8632 WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [req-8d513732-f932-47b8-bc2c-937958c30f47 - - - - -] Router 9823d34b-bb2a-480c-b3f6-cf51fd19db52 static routes [{'destination': '<a href="http://10.0.0.1/24" target="_blank">10.0.0.1/24</a>', 'nexthop': '169.254.100.1'}, {'destination': '<a href="http://10.0.2.1/24" target="_blank">10.0.2.1/24</a>', 'nexthop': '169.254.100.3'}] found in OVN but not in Neutron<br><br><br>Any suggestions on how to resolve this db_sync issue? since all other Neutron modules did not present any problem with OVN-IC (as far as I tested). <br>I remember Terry was keen to suggest some things to help. I believe that before proposing some complex mechanism to solve this simple problem, I would like to hear the community opinion. In our use case, a bit change in db_sync with filter by neutron key in external_ids would be enough.<br><br>Regards,</div><div>Roberto<br><br><br><br>[1] <a href="https://lists.openstack.org/pipermail/openstack-discuss/2023-March/032624.html" target="_blank">https://lists.openstack.org/pipermail/openstack-discuss/2023-March/032624.html</a><br>[2] <a href="https://etherpad.opendev.org/p/neutron-bobcat-ptg" target="_blank">https://etherpad.opendev.org/p/neutron-bobcat-ptg</a><br><br><br><br>Additional logs:<br><br>OpenStack 1<br><br>root@os-infra-1-neutron-ovn-northd-container-f931b37c:~# <br>root@os-infra-1-neutron-ovn-northd-container-f931b37c:~# <br>root@os-infra-1-neutron-ovn-northd-container-f931b37c:~# ovn-nbctl lr-route-list 6b776115-746a-4c59-aa73-6674c70b3498 <br>IPv4 Routes<br>Route Table <main>:<br> <a href="http://20.0.1.0/24" target="_blank">20.0.1.0/24</a> 169.254.200.2 dst-ip (learned)<br> <a href="http://20.0.2.0/24" target="_blank">20.0.2.0/24</a> 169.254.200.3 dst-ip (learned)<br> <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> 200.200.200.1 dst-ip<br><br>IPv6 Routes<br>Route Table <main>:<br> ::/0 fc00:ca5a:ca5a:8000:: dst-ip<br>root@os-infra-1-neutron-ovn-northd-container-f931b37c:~# ovn-nbctl lr-route-list 23d4552a-62c4-40e1-8bae-d06af3489c07 <br>IPv4 Routes<br>Route Table <main>:<br> <a href="http://10.0.1.0/24" target="_blank">10.0.1.0/24</a> 169.254.100.2 dst-ip (learned)<br> <a href="http://10.0.2.0/24" target="_blank">10.0.2.0/24</a> 169.254.100.3 dst-ip (learned)<br> <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> 200.200.200.1 dst-ip<br><br>IPv6 Routes<br>Route Table <main>:<br> ::/0 fc00:ca5a:ca5a:8000:: dst-ip<br>root@os-infra-1-neutron-ovn-northd-container-f931b37c:~# <br><br><br>OpenStack 2<br><br>root@os-infra-1-neutron-ovn-northd-container-30f7e935:~# ovn-nbctl lr-route-list dc1e5008-adb9-451e-8b71-09388f3680bc <br>IPv4 Routes<br>Route Table <main>:<br> <a href="http://20.0.0.0/24" target="_blank">20.0.0.0/24</a> 169.254.200.1 dst-ip (learned)<br> <a href="http://20.0.2.0/24" target="_blank">20.0.2.0/24</a> 169.254.200.3 dst-ip (learned)<br> <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> 200.200.200.1 dst-ip<br><br>IPv6 Routes<br>Route Table <main>:<br> ::/0 fc00:ca5a:ca5a:8000:: dst-ip<br>root@os-infra-1-neutron-ovn-northd-container-30f7e935:~# ovn-nbctl lr-route-list ce45f681-6454-43fe-974f-81344bb8113a <br>IPv4 Routes<br>Route Table <main>:<br> <a href="http://10.0.0.0/24" target="_blank">10.0.0.0/24</a> 169.254.100.1 dst-ip (learned)<br> <a href="http://10.0.2.0/24" target="_blank">10.0.2.0/24</a> 169.254.100.3 dst-ip (learned)<br> <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> 200.200.200.1 dst-ip<br><br>IPv6 Routes<br>Route Table <main>:<br> ::/0 fc00:ca5a:ca5a:8000:: dst-ip<br><br><br><br>OpenStack 3<br><br>root@os-infra-1-neutron-ovn-northd-container-f237db97:~# <br>root@os-infra-1-neutron-ovn-northd-container-f237db97:~# ovn-nbctl lr-route-list cfa259d6-311f-4409-bcf2-79a929835cb3 <br>IPv4 Routes<br>Route Table <main>:<br> <a href="http://20.0.0.0/24" target="_blank">20.0.0.0/24</a> 169.254.200.1 dst-ip (learned)<br> <a href="http://20.0.1.0/24" target="_blank">20.0.1.0/24</a> 169.254.200.2 dst-ip (learned)<br> <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> 200.200.200.1 dst-ip<br><br>IPv6 Routes<br>Route Table <main>:<br> ::/0 fc00:ca5a:ca5a:8000:: dst-ip<br>root@os-infra-1-neutron-ovn-northd-container-f237db97:~# ovn-nbctl lr-route-list c5a4dcd8-b9a6-4397-a7cf-88bc1e01b0b0 <br>IPv4 Routes<br>Route Table <main>:<br> <a href="http://10.0.0.0/24" target="_blank">10.0.0.0/24</a> 169.254.100.1 dst-ip (learned)<br> <a href="http://10.0.1.0/24" target="_blank">10.0.1.0/24</a> 169.254.100.2 dst-ip (learned)<br> <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> 200.200.200.1 dst-ip<br><br>IPv6 Routes<br>Route Table <main>:<br> ::/0 fc00:ca5a:ca5a:8000:: dst-ip<br><br><br>OVN-IC Global database<br><br>root@ovn-global-db1:~# ovn-ic-sbctl show<br>availability-zone osp1<br> gateway 832b6c0d-13ce-4600-ab37-78516d8ec4c5<br> hostname: osp1-gwnode1<br> type: geneve<br> ip: 192.168.200.28<br> port admin-rt1-tenant1<br> transit switch: admin-tenant1<br> address: ["aa:aa:aa:aa:bb:01 <a href="http://169.254.100.1/24" target="_blank">169.254.100.1/24</a> fe80::1/64"]<br> port admin-rt1-tenant1_1<br> transit switch: admin-tenant1_1<br> address: ["aa:aa:aa:aa:dd:01 <a href="http://169.254.200.1/24" target="_blank">169.254.200.1/24</a>"]<br>availability-zone osp2<br> gateway 17ffabdf-cf47-41ab-9539-d326c13c4ca8<br> hostname: osp2-gwnode1<br> type: geneve<br> ip: 192.168.200.128<br> port admin-rt2-tenant1<br> transit switch: admin-tenant1<br> address: ["aa:aa:aa:aa:bb:02 <a href="http://169.254.100.2/24" target="_blank">169.254.100.2/24</a> fe80::2/64"]<br> port admin-rt2-tenant1_1<br> transit switch: admin-tenant1_1<br> address: ["aa:aa:aa:aa:dd:02 <a href="http://169.254.200.2/24" target="_blank">169.254.200.2/24</a>"]<br>availability-zone osp3<br> gateway 97595af9-7896-40d0-a883-beadbff1aa5b<br> hostname: osp3-gwnode1<br> type: geneve<br> ip: 192.168.200.228<br> port admin-rt3-tenant1<br> transit switch: admin-tenant1<br> address: ["aa:aa:aa:aa:aa:03 <a href="http://169.254.100.3/24" target="_blank">169.254.100.3/24</a> fe80::3/64"]<br> port admin-rt3-tenant1_1<br> transit switch: admin-tenant1_1<br> address: ["aa:aa:aa:aa:dd:03 <a href="http://169.254.200.3/24" target="_blank">169.254.200.3/24</a>"]<br><br><br><br><br><br></div></div>
<br>
<div style="font-family:Arial,Helvetica,sans-serif;font-size:1.3em"><div style="color:rgb(97,97,97);font-family:"Open Sans";font-size:14px;line-height:21px;background-color:rgb(255,255,255)"><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;line-height:normal"><div style="font-family:Arial,Helvetica,sans-serif;font-size:1.3em"><div style="color:rgb(97,97,97);font-family:"Open Sans";font-size:14px;line-height:21px"><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;line-height:normal"><br></div></div></div><div style="font-family:Arial,Helvetica,sans-serif;font-size:1.3em"><i style="font-family:arial,sans-serif;font-size:x-small">‘Esta mensagem é direcionada apenas para os endereços constantes no cabeçalho inicial. Se você não está listado nos endereços constantes no cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão imediatamente anuladas e proibidas’.</i></div><div style="font-family:Arial,Helvetica,sans-serif;font-size:1.3em"><p style="font-family:arial,sans-serif;text-align:justify"><i><font size="1"> </font></i><i><font size="1">‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por quaisquer perdas ou danos causados por esse e-mail ou por seus anexos’.</font></i></p></div></div></div></div></blockquote></div>
</blockquote></div>
</blockquote></div></div>
<br>
<div style="font-family:Arial,Helvetica,sans-serif;font-size:1.3em"><div style="color:rgb(97,97,97);font-family:"Open Sans";font-size:14px;line-height:21px;background-color:rgb(255,255,255)"><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;line-height:normal"><div style="font-family:Arial,Helvetica,sans-serif;font-size:1.3em"><div style="color:rgb(97,97,97);font-family:"Open Sans";font-size:14px;line-height:21px"><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;line-height:normal"><br></div></div></div><div style="font-family:Arial,Helvetica,sans-serif;font-size:1.3em"><i style="font-family:arial,sans-serif;font-size:x-small">‘Esta mensagem é direcionada apenas para os endereços constantes no cabeçalho inicial. Se você não está listado nos endereços constantes no cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão imediatamente anuladas e proibidas’.</i></div><div style="font-family:Arial,Helvetica,sans-serif;font-size:1.3em"><p style="font-family:arial,sans-serif;text-align:justify"><i><font size="1"> </font></i><i><font size="1">‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por quaisquer perdas ou danos causados por esse e-mail ou por seus anexos’.</font></i></p></div></div></div></div></blockquote></div>