[neutron] unmanaged router resources - OVN interconnect
Roberto Bartzen Acosta
roberto.acosta at luizalabs.com
Mon Jul 17 13:10:41 UTC 2023
Hi Felix and Rodolfo,
Em seg., 17 de jul. de 2023 às 05:39, Rodolfo Alonso Hernandez <
ralonsoh at redhat.com> escreveu:
> Hello Felix:
>
> That's the point: the launchpad bug describes the registers created by the
> IC configuration but these registers do not have any identification that
> informs about their relationship with the IC TS.
>
> Rather than modifying the Neutron resources (a network is a generic
> resource that is used by multiple backends), I would try to create a NB-IC
> resource map and the corresponding NB registers (NAT, LRP and LRSR). If you
> can link a NB-IC TS with the corresponding NB resources, then the problem
> is fixed because knowing them we can explicitly skip them from the DB sync
> tool.
>
> If that is not the case, then please present an alternative with the
> suggested idea of creating an external network type.
>
> BTW, I don't understand why you first say that "created by tools not
> necessarily controlled by us we need to avoid changing them", but then you
> are talking about refactoring these tools. Are we talking about the tools
> that create these IC configurations? Can you share them?
>
>
I think we are talking about two different cases.
1 - This proposed RFE intends to solve the problem of removing external
resources and that is enough for the OVN-IC to work in generic scenarios:
multi tenancy, single external provider, multiple external providers, etc.
because the OVN-IC will not relate to Neutron's network topology or
capabilities.
2- Felix' suggestion is related to integrating the TS to a network on
Neutron (some kind of mapping), and in this case, we have a hybrid solution
with the TS managed by an external tool, and the management of the NAT, LRP
/ TS link with the tenant being done directly on Neutron. For this
proposal, I believe that the port filter would not be necessary because
Neutron would know the resource mappings in the OVN-NB, but it seems to me
a more complex solution.
The question about case 2 is related to which level we are talking about
modifying the generic network to map the TS. If it is a network provider
level, the discussion is one - this case would solve the
schedule/lrp-set-gateway-chassis problem as it would be a gateway port.
but if it is a tenant' network level linked to a router / no gateway port
(the discussion would be another) - does not solve the gateway port
problem, but it seems closer to what the OVN-IC proposes to do at the
network L3 interconnect level).
BTW: in case the TS is connected to an external network, we would need an
additional router for external/internet connectivity not related to the
OVN-IC, or maybe create a router with multiple gateway ports (that doesn't
exist yet)...
Regards,
Roberto
> Regards.
>
> On Mon, Jul 17, 2023 at 8:43 AM Felix Huettner <felix.huettner at mail.schwarz>
> wrote:
>
>> Hi everyone,
>>
>> On Thu, Jul 13, 2023 at 07:29:51PM -0300, Roberto Bartzen Acosta wrote:
>> > Hi Rodolfo,
>> >
>> > Em qui., 13 de jul. de 2023 às 13:45, Rodolfo Alonso Hernandez <
>> > ralonsoh at redhat.com> escreveu:
>> >
>> > > Hello Roberto:
>> > >
>> > > I think you have provided a very detailed explanation of the issue
>> but you
>> > > still didn't present any possible solution. I would ask you at least
>> for a
>> > > proposal and for specific details of what is failing in the OVN sync
>> tool.
>> > >
>> > > For example, if you add a TS between two clouds, you'll need to
>> create (1)
>> > > routers (NAT registers), (2) router ports (LRP) and (3) some static
>> routes
>> > > (LRSR). All these elements are monitored by the DB sync tool and will
>> fail
>> > > because the counterparts in the Neutron DB don't exist. I guess that
>> you
>> > > have some script or tool or at least a document to manually add these
>> > > resources; sharing it could help to find a solution.
>> > >
>> > > If these elements are manually created during the deployment phase,
>> you
>> > > can also control the information provided. I'm thinking about the
>> > > "external_ids" field; we can push some defined constant that will make
>> > > these registers transparent to the OVN DB sync tool.
>>
>> As these resources are also created by tools not necessarily controlled
>> by us we need to avoid changing them.
>> I would therefor propose to not add a "ignore" constant to the
>> "external_ids", to to rather check for the existence of the neutron keys
>> in "external_ids" (i think thats also what is described in the bug
>> below).
>>
>>
Thanks for the contribution Felix. The idea is not to add (or at least not
expect it to be externally added) any identification in the external_ids,
Neutron should only filter for DB consistency validation what it knows and
has created by Neuron -> Neutron in the external ids key.
> > >
>> > > Please check this idea and if it is feasible. In that case, please
>> add a
>> > > topic to the Neutron drivers meeting agenda [1] to discuss it.
>> > >
>> >
>> > That sounds good, thanks.
>> > https://bugs.launchpad.net/neutron/+bug/2027742
>>
>> From my perspective there is also an additional step after the above
>> REF. Currently it just makes it possible to use ovn-ic without neutron
>> killing it.
>> However if we could get neutron to treat a Transit Switch as a external
>> network we would skip the need to implement the NAT, LRP and LRSR logic
>> in custom tooling and could use the existing neutron code. This would
>> probably require creating a custom type of provider network and then
>> specifying the transit switch name in the provider options.
>>
>> Is that something that from your perspective we should take a look at
>> right away, or rather first focus und neutron not burning the transit
>> switch.
>>
>> Regards
>> Felix
>>
>> >
>> > Regards,
>> > Roberto
>> >
>> >
>> > >
>> > > Regards.
>> > >
>> > > [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers
>> > >
>> > >
>> > > On Wed, Jul 12, 2023 at 2:12 PM Roberto Bartzen Acosta <
>> > > roberto.acosta at luizalabs.com> wrote:
>> > >
>> > >>
>> > >>
>> > >> Em qua., 12 de jul. de 2023 às 09:05, Roberto Bartzen Acosta <
>> > >> roberto.acosta at luizalabs.com> escreveu:
>> > >>
>> > >>> Hi Rodolfo,
>> > >>>
>> > >>> Thanks for the feedback.
>> > >>>
>> > >>> I liked the idea of having a different network type only to hold the
>> > >>> Transit Switches and thus the LRP but it depends on some factors
>> like
>> > >>> multi-tenancy.
>> > >>> From the OVN perspective, the TS is global and will only be linked
>> to a
>> > >>> tenant when it is plugged into the tenant router port. My concern
>> here is
>> > >>> that if Neutron manages the TS, it will need to assume the dynamic
>> > >>> characteristics of this TS function, ovn-ic creates and removes the
>> TS in
>> > >>> NB_Global, in addition to plugging and removing ports in this switch
>> > >>> according to the network topology. Additionally, Neutron would
>> still need
>> > >>> to handle the learned remote routes (which are listed as static
>> routes from
>> > >>> the tenant router perspective).
>> > >>>
>> > >> sorry: NB_Global -> Northbound Database.
>> > >>
>> > >>
>> > >>> This is an open discussion, Felix can add ideas here.
>> > >>>
>> > >>> In general, it seems to me that we have no alternatives for this
>> type of
>> > >>> solution other than OVN-IC (note that ovn-bgp-agent does not learn
>> remote
>> > >>> routes at the SDN level / OVN).
>> > >>> OpenStack seems to be designed to run like a "bubble" and this
>> traffic
>> > >>> from one VM to another always needs to be routed at the FIP level
>> and
>> > >>> external routing layers.
>> > >>>
>> > >>> Regards,
>> > >>> Roberto
>> > >>>
>> > >>> Em qua., 12 de jul. de 2023 às 05:11, Rodolfo Alonso Hernandez <
>> > >>> ralonsoh at redhat.com> escreveu:
>> > >>>
>> > >>>> Hello Roberto:
>> > >>>>
>> > >>>> We talked about this possible RFE during the PTG [1] but I don't
>> > >>>> remember having any proposal. Actually what I remember (and was
>> written in
>> > >>>> the etherpad) is that you were going to present an RFE. Can you
>> explain it
>> > >>>> briefly?
>> > >>>>
>> > >>>> We also talked about the idea, proposed by Felix in the mailing
>> list,
>> > >>>> of having a different network type only to hold the Transit
>> Switches and
>> > >>>> thus the LRP. If you have a proposal, please present it.
>> > >>>>
>> > >>>> Regards.
>> > >>>>
>> > >>>> [1]https://etherpad.opendev.org/p/neutron-bobcat-ptg#L506
>> > >>>>
>> > >>>> On Tue, Jul 11, 2023 at 10:50 PM Roberto Bartzen Acosta <
>> > >>>> roberto.acosta at luizalabs.com> wrote:
>> > >>>>
>> > >>>>> Hello Neutron folks,
>> > >>>>>
>> > >>>>> Regarding the conversation started in March [1] about the use of
>> OVN
>> > >>>>> interconnect with Neutron, I would like to evolve the discussion
>> in
>> > >>>>> relation to resources allocated by OVN-IC and which are not
>> managed by
>> > >>>>> Neutron. In the March PTG [2], the problem related to the db_sync
>> tool was
>> > >>>>> presented, and a proposed solution in which Neutron does not
>> manage these
>> > >>>>> resources.
>> > >>>>>
>> > >>>>> After discussing some architectural designs with Felix/StackIT, it
>> > >>>>> seems to make sense to come up with a proposal to change the
>> mech_driver to
>> > >>>>> validate the external_ids key and not remove resources present in
>> the OVN
>> > >>>>> backend without Neutron "signature".
>> > >>>>>
>> > >>>>> The discussion here is more complex than it seems, because
>> > >>>>> architecturally Neutron does not allow us to interconnect
>> workloads in
>> > >>>>> multiple AZs (with different OpenStacks), but this could be a
>> requirement
>> > >>>>> for a high availability cloud solution (Cloud Region).
>> Additionally, this
>> > >>>>> OVN-IC solution allows interconnecting other cloud backends that
>> use OVN,
>> > >>>>> in the case of kubernetes with ovn-kube.
>> > >>>>>
>> > >>>>> We tested an OVN interconnect integrated with 3 OpenStack
>> > >>>>> installations and it worked very well !!! considering direct L3
>> traffic at
>> > >>>>> the router level between different network infrastructures.
>> > >>>>>
>> > >>>>> SYNC_REPAIR - problem
>> > >>>>>
>> > >>>>> * Static Routes (learned OVN-IC routes)
>> > >>>>> * Router Port -> Transit Switches
>> > >>>>>
>> > >>>>> Jul 10 18:34:11 os-infra-1-neutron-server-container-845157ae
>> > >>>>> neutron-server[8632]: 2023-07-10 18:34:11.343 8632 WARNING
>> > >>>>> neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync
>> > >>>>> [req-8d513732-f932-47b8-bc2c-937958c30f47 - - - - -] Router Port
>> found in
>> > >>>>> OVN but not in Neutron, port_id=rt2-admin-tenant1
>> > >>>>> Jul 10 18:34:11 os-infra-1-neutron-server-container-845157ae
>> > >>>>> neutron-server[8632]: 2023-07-10 18:34:11.343 8632 WARNING
>> > >>>>> neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync
>> > >>>>> [req-8d513732-f932-47b8-bc2c-937958c30f47 - - - - -] Router
>> > >>>>> 9823d34b-bb2a-480c-b3f6-cf51fd19db52 static routes
>> [{'destination': '
>> > >>>>> 10.0.0.1/24', 'nexthop': '169.254.100.1'}, {'destination': '
>> > >>>>> 10.0.2.1/24', 'nexthop': '169.254.100.3'}] found in OVN but not
>> in
>> > >>>>> Neutron
>> > >>>>>
>> > >>>>>
>> > >>>>> Any suggestions on how to resolve this db_sync issue? since all
>> other
>> > >>>>> Neutron modules did not present any problem with OVN-IC (as far
>> as I
>> > >>>>> tested).
>> > >>>>> I remember Terry was keen to suggest some things to help. I
>> believe
>> > >>>>> that before proposing some complex mechanism to solve this simple
>> problem,
>> > >>>>> I would like to hear the community opinion. In our use case, a
>> bit change
>> > >>>>> in db_sync with filter by neutron key in external_ids would be
>> enough.
>> > >>>>>
>> > >>>>> Regards,
>> > >>>>> Roberto
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> [1]
>> > >>>>>
>> https://lists.openstack.org/pipermail/openstack-discuss/2023-March/032624.html
>> > >>>>> [2] https://etherpad.opendev.org/p/neutron-bobcat-ptg
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> Additional logs:
>> > >>>>>
>> > >>>>> OpenStack 1
>> > >>>>>
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>> ovn-nbctl
>> > >>>>> lr-route-list 6b776115-746a-4c59-aa73-6674c70b3498
>> > >>>>> IPv4 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> 20.0.1.0/24 169.254.200.2 dst-ip
>> (learned)
>> > >>>>> 20.0.2.0/24 169.254.200.3 dst-ip
>> (learned)
>> > >>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>> > >>>>>
>> > >>>>> IPv6 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>> ovn-nbctl
>> > >>>>> lr-route-list 23d4552a-62c4-40e1-8bae-d06af3489c07
>> > >>>>> IPv4 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> 10.0.1.0/24 169.254.100.2 dst-ip
>> (learned)
>> > >>>>> 10.0.2.0/24 169.254.100.3 dst-ip
>> (learned)
>> > >>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>> > >>>>>
>> > >>>>> IPv6 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>> > >>>>>
>> > >>>>>
>> > >>>>> OpenStack 2
>> > >>>>>
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-30f7e935:~#
>> ovn-nbctl
>> > >>>>> lr-route-list dc1e5008-adb9-451e-8b71-09388f3680bc
>> > >>>>> IPv4 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> 20.0.0.0/24 169.254.200.1 dst-ip
>> (learned)
>> > >>>>> 20.0.2.0/24 169.254.200.3 dst-ip
>> (learned)
>> > >>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>> > >>>>>
>> > >>>>> IPv6 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-30f7e935:~#
>> ovn-nbctl
>> > >>>>> lr-route-list ce45f681-6454-43fe-974f-81344bb8113a
>> > >>>>> IPv4 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> 10.0.0.0/24 169.254.100.1 dst-ip
>> (learned)
>> > >>>>> 10.0.2.0/24 169.254.100.3 dst-ip
>> (learned)
>> > >>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>> > >>>>>
>> > >>>>> IPv6 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> OpenStack 3
>> > >>>>>
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f237db97:~#
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f237db97:~#
>> ovn-nbctl
>> > >>>>> lr-route-list cfa259d6-311f-4409-bcf2-79a929835cb3
>> > >>>>> IPv4 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> 20.0.0.0/24 169.254.200.1 dst-ip
>> (learned)
>> > >>>>> 20.0.1.0/24 169.254.200.2 dst-ip
>> (learned)
>> > >>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>> > >>>>>
>> > >>>>> IPv6 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f237db97:~#
>> ovn-nbctl
>> > >>>>> lr-route-list c5a4dcd8-b9a6-4397-a7cf-88bc1e01b0b0
>> > >>>>> IPv4 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> 10.0.0.0/24 169.254.100.1 dst-ip
>> (learned)
>> > >>>>> 10.0.1.0/24 169.254.100.2 dst-ip
>> (learned)
>> > >>>>> 0.0.0.0/0 200.200.200.1 dst-ip
>> > >>>>>
>> > >>>>> IPv6 Routes
>> > >>>>> Route Table <main>:
>> > >>>>> ::/0 fc00:ca5a:ca5a:8000:: dst-ip
>> > >>>>>
>> > >>>>>
>> > >>>>> OVN-IC Global database
>> > >>>>>
>> > >>>>> root at ovn-global-db1:~# ovn-ic-sbctl show
>> > >>>>> availability-zone osp1
>> > >>>>> gateway 832b6c0d-13ce-4600-ab37-78516d8ec4c5
>> > >>>>> hostname: osp1-gwnode1
>> > >>>>> type: geneve
>> > >>>>> ip: 192.168.200.28
>> > >>>>> port admin-rt1-tenant1
>> > >>>>> transit switch: admin-tenant1
>> > >>>>> address: ["aa:aa:aa:aa:bb:01 169.254.100.1/24
>> fe80::1/64"]
>> > >>>>> port admin-rt1-tenant1_1
>> > >>>>> transit switch: admin-tenant1_1
>> > >>>>> address: ["aa:aa:aa:aa:dd:01 169.254.200.1/24"]
>> > >>>>> availability-zone osp2
>> > >>>>> gateway 17ffabdf-cf47-41ab-9539-d326c13c4ca8
>> > >>>>> hostname: osp2-gwnode1
>> > >>>>> type: geneve
>> > >>>>> ip: 192.168.200.128
>> > >>>>> port admin-rt2-tenant1
>> > >>>>> transit switch: admin-tenant1
>> > >>>>> address: ["aa:aa:aa:aa:bb:02 169.254.100.2/24
>> fe80::2/64"]
>> > >>>>> port admin-rt2-tenant1_1
>> > >>>>> transit switch: admin-tenant1_1
>> > >>>>> address: ["aa:aa:aa:aa:dd:02 169.254.200.2/24"]
>> > >>>>> availability-zone osp3
>> > >>>>> gateway 97595af9-7896-40d0-a883-beadbff1aa5b
>> > >>>>> hostname: osp3-gwnode1
>> > >>>>> type: geneve
>> > >>>>> ip: 192.168.200.228
>> > >>>>> port admin-rt3-tenant1
>> > >>>>> transit switch: admin-tenant1
>> > >>>>> address: ["aa:aa:aa:aa:aa:03 169.254.100.3/24
>> fe80::3/64"]
>> > >>>>> port admin-rt3-tenant1_1
>> > >>>>> transit switch: admin-tenant1_1
>> > >>>>> address: ["aa:aa:aa:aa:dd:03 169.254.200.3/24"]
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> *‘Esta mensagem é direcionada apenas para os endereços constantes
>> no
>> > >>>>> cabeçalho inicial. Se você não está listado nos endereços
>> constantes no
>> > >>>>> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo
>> dessa
>> > >>>>> mensagem e cuja cópia, encaminhamento e/ou execução das ações
>> citadas estão
>> > >>>>> imediatamente anuladas e proibidas’.*
>> > >>>>>
>> > >>>>> *‘Apesar do Magazine Luiza tomar todas as precauções razoáveis
>> para
>> > >>>>> assegurar que nenhum vírus esteja presente nesse e-mail, a
>> empresa não
>> > >>>>> poderá aceitar a responsabilidade por quaisquer perdas ou danos
>> causados
>> > >>>>> por esse e-mail ou por seus anexos’.*
>> > >>>>>
>> > >>>>
>> > >>
>> > >> *‘Esta mensagem é direcionada apenas para os endereços constantes no
>> > >> cabeçalho inicial. Se você não está listado nos endereços constantes
>> no
>> > >> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo
>> dessa
>> > >> mensagem e cuja cópia, encaminhamento e/ou execução das ações
>> citadas estão
>> > >> imediatamente anuladas e proibidas’.*
>> > >>
>> > >> *‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para
>> > >> assegurar que nenhum vírus esteja presente nesse e-mail, a empresa
>> não
>> > >> poderá aceitar a responsabilidade por quaisquer perdas ou danos
>> causados
>> > >> por esse e-mail ou por seus anexos’.*
>> > >>
>> > >
>> >
>> > --
>> >
>> >
>> >
>> >
>> > _‘Esta mensagem é direcionada apenas para os endereços constantes no
>> > cabeçalho inicial. Se você não está listado nos endereços constantes no
>> > cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
>> > mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas
>> estão
>> > imediatamente anuladas e proibidas’._
>> >
>> >
>> > * **‘Apesar do Magazine Luiza tomar
>> > todas as precauções razoáveis para assegurar que nenhum vírus esteja
>> > presente nesse e-mail, a empresa não poderá aceitar a responsabilidade
>> por
>> > quaisquer perdas ou danos causados por esse e-mail ou por seus anexos’.*
>> >
>> >
>> >
>> Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für
>> die Verwertung durch den vorgesehenen Empfänger bestimmt.
>> Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender
>> bitte unverzüglich in Kenntnis und löschen diese E Mail.
>>
>> Hinweise zum Datenschutz finden Sie hier<https://www.datenschutz.schwarz
>> >.
>>
>>
>> This e-mail may contain confidential content and is intended only for the
>> specified recipient/s.
>> If you are not the intended recipient, please inform the sender
>> immediately and delete this e-mail.
>>
>> Information on data protection can be found here<
>> https://www.datenschutz.schwarz>.
>>
>>
--
_‘Esta mensagem é direcionada apenas para os endereços constantes no
cabeçalho inicial. Se você não está listado nos endereços constantes no
cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
imediatamente anuladas e proibidas’._
* **‘Apesar do Magazine Luiza tomar
todas as precauções razoáveis para assegurar que nenhum vírus esteja
presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por
quaisquer perdas ou danos causados por esse e-mail ou por seus anexos’.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230717/db9e6be1/attachment-0001.htm>
More information about the openstack-discuss
mailing list