[neutron] unmanaged router resources - OVN interconnect
Roberto Bartzen Acosta
roberto.acosta at luizalabs.com
Thu Jul 13 22:29:51 UTC 2023
Hi Rodolfo,
Em qui., 13 de jul. de 2023 às 13:45, Rodolfo Alonso Hernandez <
ralonsoh at redhat.com> escreveu:
> Hello Roberto:
>
> I think you have provided a very detailed explanation of the issue but you
> still didn't present any possible solution. I would ask you at least for a
> proposal and for specific details of what is failing in the OVN sync tool.
>
> For example, if you add a TS between two clouds, you'll need to create (1)
> routers (NAT registers), (2) router ports (LRP) and (3) some static routes
> (LRSR). All these elements are monitored by the DB sync tool and will fail
> because the counterparts in the Neutron DB don't exist. I guess that you
> have some script or tool or at least a document to manually add these
> resources; sharing it could help to find a solution.
>
> If these elements are manually created during the deployment phase, you
> can also control the information provided. I'm thinking about the
> "external_ids" field; we can push some defined constant that will make
> these registers transparent to the OVN DB sync tool.
>
> Please check this idea and if it is feasible. In that case, please add a
> topic to the Neutron drivers meeting agenda [1] to discuss it.
>
That sounds good, thanks.
https://bugs.launchpad.net/neutron/+bug/2027742
Regards,
Roberto
>
> Regards.
>
> [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers
>
>
> On Wed, Jul 12, 2023 at 2:12 PM Roberto Bartzen Acosta <
> roberto.acosta at luizalabs.com> wrote:
>
>>
>>
>> Em qua., 12 de jul. de 2023 às 09:05, Roberto Bartzen Acosta <
>> roberto.acosta at luizalabs.com> escreveu:
>>
>>> Hi Rodolfo,
>>>
>>> Thanks for the feedback.
>>>
>>> I liked the idea of having a different network type only to hold the
>>> Transit Switches and thus the LRP but it depends on some factors like
>>> multi-tenancy.
>>> From the OVN perspective, the TS is global and will only be linked to a
>>> tenant when it is plugged into the tenant router port. My concern here is
>>> that if Neutron manages the TS, it will need to assume the dynamic
>>> characteristics of this TS function, ovn-ic creates and removes the TS in
>>> NB_Global, in addition to plugging and removing ports in this switch
>>> according to the network topology. Additionally, Neutron would still need
>>> to handle the learned remote routes (which are listed as static routes from
>>> the tenant router perspective).
>>>
>> sorry: NB_Global -> Northbound Database.
>>
>>
>>> This is an open discussion, Felix can add ideas here.
>>>
>>> In general, it seems to me that we have no alternatives for this type of
>>> solution other than OVN-IC (note that ovn-bgp-agent does not learn remote
>>> routes at the SDN level / OVN).
>>> OpenStack seems to be designed to run like a "bubble" and this traffic
>>> from one VM to another always needs to be routed at the FIP level and
>>> external routing layers.
>>>
>>> Regards,
>>> Roberto
>>>
>>> Em qua., 12 de jul. de 2023 às 05:11, Rodolfo Alonso Hernandez <
>>> ralonsoh at redhat.com> escreveu:
>>>
>>>> Hello Roberto:
>>>>
>>>> We talked about this possible RFE during the PTG [1] but I don't
>>>> remember having any proposal. Actually what I remember (and was written in
>>>> the etherpad) is that you were going to present an RFE. Can you explain it
>>>> briefly?
>>>>
>>>> We also talked about the idea, proposed by Felix in the mailing list,
>>>> of having a different network type only to hold the Transit Switches and
>>>> thus the LRP. If you have a proposal, please present it.
>>>>
>>>> Regards.
>>>>
>>>> [1]https://etherpad.opendev.org/p/neutron-bobcat-ptg#L506
>>>>
>>>> On Tue, Jul 11, 2023 at 10:50 PM Roberto Bartzen Acosta <
>>>> roberto.acosta at luizalabs.com> wrote:
>>>>
>>>>> Hello Neutron folks,
>>>>>
>>>>> Regarding the conversation started in March [1] about the use of OVN
>>>>> interconnect with Neutron, I would like to evolve the discussion in
>>>>> relation to resources allocated by OVN-IC and which are not managed by
>>>>> Neutron. In the March PTG [2], the problem related to the db_sync tool was
>>>>> presented, and a proposed solution in which Neutron does not manage these
>>>>> resources.
>>>>>
>>>>> After discussing some architectural designs with Felix/StackIT, it
>>>>> seems to make sense to come up with a proposal to change the mech_driver to
>>>>> validate the external_ids key and not remove resources present in the OVN
>>>>> backend without Neutron "signature".
>>>>>
>>>>> The discussion here is more complex than it seems, because
>>>>> architecturally Neutron does not allow us to interconnect workloads in
>>>>> multiple AZs (with different OpenStacks), but this could be a requirement
>>>>> for a high availability cloud solution (Cloud Region). Additionally, this
>>>>> OVN-IC solution allows interconnecting other cloud backends that use OVN,
>>>>> in the case of kubernetes with ovn-kube.
>>>>>
>>>>> We tested an OVN interconnect integrated with 3 OpenStack
>>>>> installations and it worked very well !!! considering direct L3 traffic at
>>>>> the router level between different network infrastructures.
>>>>>
>>>>> SYNC_REPAIR - problem
>>>>>
>>>>> * Static Routes (learned OVN-IC routes)
>>>>> * Router Port -> Transit Switches
>>>>>
>>>>> Jul 10 18:34:11 os-infra-1-neutron-server-container-845157ae
>>>>> neutron-server[8632]: 2023-07-10 18:34:11.343 8632 WARNING
>>>>> neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync
>>>>> [req-8d513732-f932-47b8-bc2c-937958c30f47 - - - - -] Router Port found in
>>>>> OVN but not in Neutron, port_id=rt2-admin-tenant1
>>>>> Jul 10 18:34:11 os-infra-1-neutron-server-container-845157ae
>>>>> neutron-server[8632]: 2023-07-10 18:34:11.343 8632 WARNING
>>>>> neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync
>>>>> [req-8d513732-f932-47b8-bc2c-937958c30f47 - - - - -] Router
>>>>> 9823d34b-bb2a-480c-b3f6-cf51fd19db52 static routes [{'destination': '
>>>>> 10.0.0.1/24', 'nexthop': '169.254.100.1'}, {'destination': '
>>>>> 10.0.2.1/24', 'nexthop': '169.254.100.3'}] found in OVN but not in
>>>>> Neutron
>>>>>
>>>>>
>>>>> Any suggestions on how to resolve this db_sync issue? since all other
>>>>> Neutron modules did not present any problem with OVN-IC (as far as I
>>>>> tested).
>>>>> I remember Terry was keen to suggest some things to help. I believe
>>>>> that before proposing some complex mechanism to solve this simple problem,
>>>>> I would like to hear the community opinion. In our use case, a bit change
>>>>> in db_sync with filter by neutron key in external_ids would be enough.
>>>>>
>>>>> Regards,
>>>>> Roberto
>>>>>
>>>>>
>>>>>
>>>>> [1]
>>>>> https://lists.openstack.org/pipermail/openstack-discuss/2023-March/032624.html
>>>>> [2] https://etherpad.opendev.org/p/neutron-bobcat-ptg
>>>>>
>>>>>
>>>>>
>>>>> Additional logs:
>>>>>
>>>>> OpenStack 1
>>>>>
>>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~# ovn-nbctl
>>>>> lr-route-list 6b776115-746a-4c59-aa73-6674c70b3498
>>>>> IPv4 Routes
>>>>> Route Table <main>:
>>>>> 20.0.1.0/24 169.254.200.2 dst-ip (learned)
>>>>> 20.0.2.0/24 169.254.200.3 dst-ip (learned)
>>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>>>>>
>>>>> IPv6 Routes
>>>>> Route Table <main>:
>>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~# ovn-nbctl
>>>>> lr-route-list 23d4552a-62c4-40e1-8bae-d06af3489c07
>>>>> IPv4 Routes
>>>>> Route Table <main>:
>>>>> 10.0.1.0/24 169.254.100.2 dst-ip (learned)
>>>>> 10.0.2.0/24 169.254.100.3 dst-ip (learned)
>>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>>>>>
>>>>> IPv6 Routes
>>>>> Route Table <main>:
>>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>>>>>
>>>>>
>>>>> OpenStack 2
>>>>>
>>>>> root at os-infra-1-neutron-ovn-northd-container-30f7e935:~# ovn-nbctl
>>>>> lr-route-list dc1e5008-adb9-451e-8b71-09388f3680bc
>>>>> IPv4 Routes
>>>>> Route Table <main>:
>>>>> 20.0.0.0/24 169.254.200.1 dst-ip (learned)
>>>>> 20.0.2.0/24 169.254.200.3 dst-ip (learned)
>>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>>>>>
>>>>> IPv6 Routes
>>>>> Route Table <main>:
>>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>>>>> root at os-infra-1-neutron-ovn-northd-container-30f7e935:~# ovn-nbctl
>>>>> lr-route-list ce45f681-6454-43fe-974f-81344bb8113a
>>>>> IPv4 Routes
>>>>> Route Table <main>:
>>>>> 10.0.0.0/24 169.254.100.1 dst-ip (learned)
>>>>> 10.0.2.0/24 169.254.100.3 dst-ip (learned)
>>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>>>>>
>>>>> IPv6 Routes
>>>>> Route Table <main>:
>>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>>>>>
>>>>>
>>>>>
>>>>> OpenStack 3
>>>>>
>>>>> root at os-infra-1-neutron-ovn-northd-container-f237db97:~#
>>>>> root at os-infra-1-neutron-ovn-northd-container-f237db97:~# ovn-nbctl
>>>>> lr-route-list cfa259d6-311f-4409-bcf2-79a929835cb3
>>>>> IPv4 Routes
>>>>> Route Table <main>:
>>>>> 20.0.0.0/24 169.254.200.1 dst-ip (learned)
>>>>> 20.0.1.0/24 169.254.200.2 dst-ip (learned)
>>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>>>>>
>>>>> IPv6 Routes
>>>>> Route Table <main>:
>>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>>>>> root at os-infra-1-neutron-ovn-northd-container-f237db97:~# ovn-nbctl
>>>>> lr-route-list c5a4dcd8-b9a6-4397-a7cf-88bc1e01b0b0
>>>>> IPv4 Routes
>>>>> Route Table <main>:
>>>>> 10.0.0.0/24 169.254.100.1 dst-ip (learned)
>>>>> 10.0.1.0/24 169.254.100.2 dst-ip (learned)
>>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>>>>>
>>>>> IPv6 Routes
>>>>> Route Table <main>:
>>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>>>>>
>>>>>
>>>>> OVN-IC Global database
>>>>>
>>>>> root at ovn-global-db1:~# ovn-ic-sbctl show
>>>>> availability-zone osp1
>>>>> gateway 832b6c0d-13ce-4600-ab37-78516d8ec4c5
>>>>> hostname: osp1-gwnode1
>>>>> type: geneve
>>>>> ip: 192.168.200.28
>>>>> port admin-rt1-tenant1
>>>>> transit switch: admin-tenant1
>>>>> address: ["aa:aa:aa:aa:bb:01 169.254.100.1/24 fe80::1/64"]
>>>>> port admin-rt1-tenant1_1
>>>>> transit switch: admin-tenant1_1
>>>>> address: ["aa:aa:aa:aa:dd:01 169.254.200.1/24"]
>>>>> availability-zone osp2
>>>>> gateway 17ffabdf-cf47-41ab-9539-d326c13c4ca8
>>>>> hostname: osp2-gwnode1
>>>>> type: geneve
>>>>> ip: 192.168.200.128
>>>>> port admin-rt2-tenant1
>>>>> transit switch: admin-tenant1
>>>>> address: ["aa:aa:aa:aa:bb:02 169.254.100.2/24 fe80::2/64"]
>>>>> port admin-rt2-tenant1_1
>>>>> transit switch: admin-tenant1_1
>>>>> address: ["aa:aa:aa:aa:dd:02 169.254.200.2/24"]
>>>>> availability-zone osp3
>>>>> gateway 97595af9-7896-40d0-a883-beadbff1aa5b
>>>>> hostname: osp3-gwnode1
>>>>> type: geneve
>>>>> ip: 192.168.200.228
>>>>> port admin-rt3-tenant1
>>>>> transit switch: admin-tenant1
>>>>> address: ["aa:aa:aa:aa:aa:03 169.254.100.3/24 fe80::3/64"]
>>>>> port admin-rt3-tenant1_1
>>>>> transit switch: admin-tenant1_1
>>>>> address: ["aa:aa:aa:aa:dd:03 169.254.200.3/24"]
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *‘Esta mensagem é direcionada apenas para os endereços constantes no
>>>>> cabeçalho inicial. Se você não está listado nos endereços constantes no
>>>>> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
>>>>> mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
>>>>> imediatamente anuladas e proibidas’.*
>>>>>
>>>>> *‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para
>>>>> assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não
>>>>> poderá aceitar a responsabilidade por quaisquer perdas ou danos causados
>>>>> por esse e-mail ou por seus anexos’.*
>>>>>
>>>>
>>
>> *‘Esta mensagem é direcionada apenas para os endereços constantes no
>> cabeçalho inicial. Se você não está listado nos endereços constantes no
>> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
>> mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
>> imediatamente anuladas e proibidas’.*
>>
>> *‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para
>> assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não
>> poderá aceitar a responsabilidade por quaisquer perdas ou danos causados
>> por esse e-mail ou por seus anexos’.*
>>
>
--
_‘Esta mensagem é direcionada apenas para os endereços constantes no
cabeçalho inicial. Se você não está listado nos endereços constantes no
cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
imediatamente anuladas e proibidas’._
* **‘Apesar do Magazine Luiza tomar
todas as precauções razoáveis para assegurar que nenhum vírus esteja
presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por
quaisquer perdas ou danos causados por esse e-mail ou por seus anexos’.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230713/063526fd/attachment-0001.htm>
More information about the openstack-discuss
mailing list