Open Stack

> In my opinion the answer would be: yes and no, this approach would work
for isolated cases, but think on a larger scale: how many external networks
would need to be managed at the IP Fabric level?
This is exactly what you are proposing in your previous mails: "Felix'
suggestion is related to integrating the TS to a network on Neutron (some
kind of mapping)". So at this point I'm curious about your proposal. And
about the static routes, if these routes are created from a Neutron defined
router, as I proposed, they won't be deleted.

> About the proposed RFE, do you see any problem if Neutron deletes from
the OVN only what it manages?
Yes, a huge one: OVN is a service for Neutron, same as RabbitMQ or MySQL.
These services are 100% controlled by Neutron because the resources can
clash. You can't manually create an IPAM register in the DB because that
will disable one IP address. The same for OVN: you can't manually create a
LRP in a Logical Switch that provides a Neutron network because this LRP
will have an IP address. If a new Neutron port is created and it uses the
same IP address of the existing LRP (that doesn't exist in Neutron), the
new port will be considered virtual and won't be able to be used in a VM
(this is an actual problem I found in a customer last week). So no, Neutron
should not allow other resources in OVN apart from what Neutron creates.

On Mon, Jul 17, 2023 at 6:13 PM Roberto Bartzen Acosta <
roberto.acosta at luizalabs.com> wrote:

> Hi Rodolfo,
>
> Em seg., 17 de jul. de 2023 às 11:30, Rodolfo Alonso Hernandez <
> ralonsoh at redhat.com> escreveu:
>
>> Ok, I'll ask again for the tool/script/manual used to make the IC
>> connection. I will also ask again for the NB-IC database resources and the
>> NB resources needed, and how are related.
>>
>
> To be clear, this tool doesn't exist yet and we are working on the design
> of the software solution [1]. For now, the tests were performed using
> manual commands, such as those described in the RFE link and very similar
> to the case applied by the ovn-kube to use OVN-IC with OpenStack - Neutron
> [2].
> The main idea is to develop an external layer interface with an API/SDK
> and an internal layer interface with the OVN-IC-NB global database and the
> OpenStack/AZ OVN-NB database [1].
>
> [1]
> https://drive.google.com/file/d/1yx66YcxNMH_oB11--0-_0eZzmBJHu-Ox/view?usp=sharing
> [2]
> https://github.com/kubeovn/kube-ovn/blob/v1.11.0/docs/cluster-interconnection.md
>
>
>> If you want Neutron to handle these resources, then you don't need
>> anything special: you can add an external network, handled only by one
>> project. Inside this project you can create the needed resources: networks,
>> routers, etc. This project/user (controlled by the sysadmin), won't create
>> any VM on these networks/routers. Then you can use these resources to
>> connect the TS.
>>
>
> In my opinion the answer would be: yes and no, this approach would work
> for isolated cases, but think on a larger scale: how many external networks
> would need to be managed at the IP Fabric level? what is the service type?
> vlan (we would have segmentation limits) vxlan (we would have to extend the
> L2 domain across the entire IP fabric) virtual? . I don't know, there are
> many questions here. Felix can explain better and propose an RFE in the
> future. Even so, Neutron would need a filter to not delete the static
> routes learned by the router (the different proposals can be
> complementary...)
>
>
> About the proposed RFE, do you see any problem if Neutron deletes from the
> OVN only what it manages?
>
>
>>
>> On Mon, Jul 17, 2023 at 3:10 PM Roberto Bartzen Acosta <
>> roberto.acosta at luizalabs.com> wrote:
>>
>>> Hi Felix and Rodolfo,
>>>
>>>
>>> Em seg., 17 de jul. de 2023 às 05:39, Rodolfo Alonso Hernandez <
>>> ralonsoh at redhat.com> escreveu:
>>>
>>>> Hello Felix:
>>>>
>>>> That's the point: the launchpad bug describes the registers created by
>>>> the IC configuration but these registers do not have any identification
>>>> that informs about their relationship with the IC TS.
>>>>
>>>> Rather than modifying the Neutron resources (a network is a generic
>>>> resource that is used by multiple backends), I would try to create a NB-IC
>>>> resource map and the corresponding NB registers (NAT, LRP and LRSR). If you
>>>> can link a NB-IC TS with the corresponding NB resources, then the problem
>>>> is fixed because knowing them we can explicitly skip them from the DB sync
>>>> tool.
>>>>
>>>> If that is not the case, then please present an alternative with the
>>>> suggested idea of creating an external network type.
>>>>
>>>> BTW, I don't understand why you first say that "created by tools not
>>>> necessarily controlled by us we need to avoid changing them", but then you
>>>> are talking about refactoring these tools. Are we talking about the tools
>>>> that create these IC configurations? Can you share them?
>>>>
>>>>
>>> I think we are talking about two different cases.
>>>
>>> 1 - This proposed RFE intends to solve the problem of removing external
>>> resources and that is enough for the OVN-IC to work in generic scenarios:
>>> multi tenancy, single external provider, multiple external providers, etc.
>>> because the OVN-IC will not relate to Neutron's network topology or
>>> capabilities.
>>>
>>> 2- Felix' suggestion is related to integrating the TS to a network on
>>> Neutron (some kind of mapping), and in this case, we have a hybrid solution
>>> with the TS managed by an external tool, and the management of the NAT, LRP
>>> / TS link with the tenant being done directly on Neutron. For this
>>> proposal, I believe that the port filter would not be necessary because
>>> Neutron would know the resource mappings in the OVN-NB, but it seems to me
>>> a more complex solution.
>>>
>>> The question about case 2 is related to which level we are talking about
>>> modifying the generic network to map the TS. If it is a network provider
>>> level, the discussion is one - this case would solve the
>>> schedule/lrp-set-gateway-chassis problem as it would be a gateway port.
>>> but if it is a tenant' network level linked to a router / no gateway
>>> port (the discussion would be another) - does not solve the gateway port
>>> problem, but it seems closer to what the OVN-IC proposes to do at the
>>> network L3 interconnect level).
>>>
>>> BTW: in case the TS is connected to an external network, we would need
>>> an additional router for external/internet connectivity not related to the
>>> OVN-IC, or maybe create a router with multiple gateway ports (that doesn't
>>> exist yet)...
>>>
>>> Regards,
>>> Roberto
>>>
>>>
>>>
>>>> Regards.
>>>>
>>>> On Mon, Jul 17, 2023 at 8:43 AM Felix Huettner
>>>> <felix.huettner at mail.schwarz> wrote:
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> On Thu, Jul 13, 2023 at 07:29:51PM -0300, Roberto Bartzen Acosta wrote:
>>>>> > Hi Rodolfo,
>>>>> >
>>>>> > Em qui., 13 de jul. de 2023 às 13:45, Rodolfo Alonso Hernandez <
>>>>> > ralonsoh at redhat.com> escreveu:
>>>>> >
>>>>> > > Hello Roberto:
>>>>> > >
>>>>> > > I think you have provided a very detailed explanation of the issue
>>>>> but you
>>>>> > > still didn't present any possible solution. I would ask you at
>>>>> least for a
>>>>> > > proposal and for specific details of what is failing in the OVN
>>>>> sync tool.
>>>>> > >
>>>>> > > For example, if you add a TS between two clouds, you'll need to
>>>>> create (1)
>>>>> > > routers (NAT registers), (2) router ports (LRP) and (3) some
>>>>> static routes
>>>>> > > (LRSR). All these elements are monitored by the DB sync tool and
>>>>> will fail
>>>>> > > because the counterparts in the Neutron DB don't exist. I guess
>>>>> that you
>>>>> > > have some script or tool or at least a document to manually add
>>>>> these
>>>>> > > resources; sharing it could help to find a solution.
>>>>> > >
>>>>> > > If these elements are manually created during the deployment
>>>>> phase, you
>>>>> > > can also control the information provided. I'm thinking about the
>>>>> > > "external_ids" field; we can push some defined constant that will
>>>>> make
>>>>> > > these registers transparent to the OVN DB sync tool.
>>>>>
>>>>> As these resources are also created by tools not necessarily controlled
>>>>> by us we need to avoid changing them.
>>>>> I would therefor propose to not add a "ignore" constant to the
>>>>> "external_ids", to to rather check for the existence of the neutron
>>>>> keys
>>>>> in "external_ids" (i think thats also what is described in the bug
>>>>> below).
>>>>>
>>>>>
>>> Thanks for the contribution Felix. The idea is not to add (or at least
>>> not expect it to be externally added) any identification in the
>>> external_ids, Neutron should only filter for DB consistency validation what
>>> it knows and has created by Neuron -> Neutron in the external ids key.
>>>
>>>
>>>> > >
>>>>> > > Please check this idea and if it is feasible. In that case, please
>>>>> add a
>>>>> > > topic to the Neutron drivers meeting agenda [1] to discuss it.
>>>>> > >
>>>>> >
>>>>> > That sounds good, thanks.
>>>>> > https://bugs.launchpad.net/neutron/+bug/2027742
>>>>>
>>>>> From my perspective there is also an additional step after the above
>>>>> REF. Currently it just makes it possible to use ovn-ic without neutron
>>>>> killing it.
>>>>> However if we could get neutron to treat a Transit Switch as a external
>>>>> network we would skip the need to implement the NAT, LRP and LRSR logic
>>>>> in custom tooling and could use the existing neutron code. This would
>>>>> probably require creating a custom type of provider network and then
>>>>> specifying the transit switch name in the provider options.
>>>>>
>>>>> Is that something that from your perspective we should take a look at
>>>>> right away, or rather first focus und neutron not burning the transit
>>>>> switch.
>>>>>
>>>>> Regards
>>>>> Felix
>>>>>
>>>>> >
>>>>> > Regards,
>>>>> > Roberto
>>>>> >
>>>>> >
>>>>> > >
>>>>> > > Regards.
>>>>> > >
>>>>> > > [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers
>>>>> > >
>>>>> > >
>>>>> > > On Wed, Jul 12, 2023 at 2:12 PM Roberto Bartzen Acosta <
>>>>> > > roberto.acosta at luizalabs.com> wrote:
>>>>> > >
>>>>> > >>
>>>>> > >>
>>>>> > >> Em qua., 12 de jul. de 2023 às 09:05, Roberto Bartzen Acosta <
>>>>> > >> roberto.acosta at luizalabs.com> escreveu:
>>>>> > >>
>>>>> > >>> Hi Rodolfo,
>>>>> > >>>
>>>>> > >>> Thanks for the feedback.
>>>>> > >>>
>>>>> > >>> I liked the idea of having a different network type only to hold
>>>>> the
>>>>> > >>> Transit Switches and thus the LRP but it depends on some factors
>>>>> like
>>>>> > >>> multi-tenancy.
>>>>> > >>> From the OVN perspective, the TS is global and will only be
>>>>> linked to a
>>>>> > >>> tenant when it is plugged into the tenant router port. My
>>>>> concern here is
>>>>> > >>> that if Neutron manages the TS, it will need to assume the
>>>>> dynamic
>>>>> > >>> characteristics of this TS function, ovn-ic creates and removes
>>>>> the TS in
>>>>> > >>> NB_Global, in addition to plugging and removing ports in this
>>>>> switch
>>>>> > >>> according to the network topology. Additionally, Neutron would
>>>>> still need
>>>>> > >>> to handle the learned remote routes (which are listed as static
>>>>> routes from
>>>>> > >>> the tenant router perspective).
>>>>> > >>>
>>>>> > >> sorry: NB_Global  -> Northbound Database.
>>>>> > >>
>>>>> > >>
>>>>> > >>> This is an open discussion, Felix can add ideas here.
>>>>> > >>>
>>>>> > >>> In general, it seems to me that we have no alternatives for this
>>>>> type of
>>>>> > >>> solution other than OVN-IC (note that ovn-bgp-agent does not
>>>>> learn remote
>>>>> > >>> routes at the SDN level / OVN).
>>>>> > >>> OpenStack seems to be designed to run like a "bubble" and this
>>>>> traffic
>>>>> > >>> from one VM to another always needs to be routed at the FIP
>>>>> level and
>>>>> > >>> external routing layers.
>>>>> > >>>
>>>>> > >>> Regards,
>>>>> > >>> Roberto
>>>>> > >>>
>>>>> > >>> Em qua., 12 de jul. de 2023 às 05:11, Rodolfo Alonso Hernandez <
>>>>> > >>> ralonsoh at redhat.com> escreveu:
>>>>> > >>>
>>>>> > >>>> Hello Roberto:
>>>>> > >>>>
>>>>> > >>>> We talked about this possible RFE during the PTG [1] but I don't
>>>>> > >>>> remember having any proposal. Actually what I remember (and was
>>>>> written in
>>>>> > >>>> the etherpad) is that you were going to present an RFE. Can you
>>>>> explain it
>>>>> > >>>> briefly?
>>>>> > >>>>
>>>>> > >>>> We also talked about the idea, proposed by Felix in the mailing
>>>>> list,
>>>>> > >>>> of having a different network type only to hold the Transit
>>>>> Switches and
>>>>> > >>>> thus the LRP. If you have a proposal, please present it.
>>>>> > >>>>
>>>>> > >>>> Regards.
>>>>> > >>>>
>>>>> > >>>> [1]https://etherpad.opendev.org/p/neutron-bobcat-ptg#L506
>>>>> > >>>>
>>>>> > >>>> On Tue, Jul 11, 2023 at 10:50 PM Roberto Bartzen Acosta <
>>>>> > >>>> roberto.acosta at luizalabs.com> wrote:
>>>>> > >>>>
>>>>> > >>>>> Hello Neutron folks,
>>>>> > >>>>>
>>>>> > >>>>> Regarding the conversation started in March [1] about the use
>>>>> of OVN
>>>>> > >>>>> interconnect with Neutron, I would like to evolve the
>>>>> discussion in
>>>>> > >>>>> relation to resources allocated by OVN-IC and which are not
>>>>> managed by
>>>>> > >>>>> Neutron. In the March PTG [2], the problem related to the
>>>>> db_sync tool was
>>>>> > >>>>> presented, and a proposed solution in which Neutron does not
>>>>> manage these
>>>>> > >>>>> resources.
>>>>> > >>>>>
>>>>> > >>>>> After discussing some architectural designs with
>>>>> Felix/StackIT, it
>>>>> > >>>>> seems to make sense to come up with a proposal to change the
>>>>> mech_driver to
>>>>> > >>>>> validate the external_ids key and not remove resources present
>>>>> in the OVN
>>>>> > >>>>> backend without Neutron "signature".
>>>>> > >>>>>
>>>>> > >>>>> The discussion here is more complex than it seems, because
>>>>> > >>>>> architecturally Neutron does not allow us to interconnect
>>>>> workloads in
>>>>> > >>>>> multiple AZs (with different OpenStacks), but this could be a
>>>>> requirement
>>>>> > >>>>> for a high availability cloud solution (Cloud Region).
>>>>> Additionally, this
>>>>> > >>>>> OVN-IC solution allows interconnecting other cloud backends
>>>>> that use OVN,
>>>>> > >>>>> in the case of kubernetes with ovn-kube.
>>>>> > >>>>>
>>>>> > >>>>> We tested an OVN interconnect integrated with 3 OpenStack
>>>>> > >>>>> installations and it worked very well !!! considering direct
>>>>> L3 traffic at
>>>>> > >>>>> the router level between different network infrastructures.
>>>>> > >>>>>
>>>>> > >>>>> SYNC_REPAIR - problem
>>>>> > >>>>>
>>>>> > >>>>> * Static Routes (learned OVN-IC routes)
>>>>> > >>>>> * Router Port -> Transit Switches
>>>>> > >>>>>
>>>>> > >>>>> Jul 10 18:34:11 os-infra-1-neutron-server-container-845157ae
>>>>> > >>>>> neutron-server[8632]: 2023-07-10 18:34:11.343 8632 WARNING
>>>>> > >>>>> neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync
>>>>> > >>>>> [req-8d513732-f932-47b8-bc2c-937958c30f47 - - - - -] Router
>>>>> Port found in
>>>>> > >>>>> OVN but not in Neutron, port_id=rt2-admin-tenant1
>>>>> > >>>>> Jul 10 18:34:11 os-infra-1-neutron-server-container-845157ae
>>>>> > >>>>> neutron-server[8632]: 2023-07-10 18:34:11.343 8632 WARNING
>>>>> > >>>>> neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync
>>>>> > >>>>> [req-8d513732-f932-47b8-bc2c-937958c30f47 - - - - -] Router
>>>>> > >>>>> 9823d34b-bb2a-480c-b3f6-cf51fd19db52 static routes
>>>>> [{'destination': '
>>>>> > >>>>> 10.0.0.1/24', 'nexthop': '169.254.100.1'}, {'destination': '
>>>>> > >>>>> 10.0.2.1/24', 'nexthop': '169.254.100.3'}] found in OVN but
>>>>> not in
>>>>> > >>>>> Neutron
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> Any suggestions on how to resolve this db_sync issue? since
>>>>> all other
>>>>> > >>>>> Neutron modules did not present any problem with OVN-IC (as
>>>>> far as I
>>>>> > >>>>> tested).
>>>>> > >>>>> I remember Terry was keen to suggest some things to help. I
>>>>> believe
>>>>> > >>>>> that before proposing some complex mechanism to solve this
>>>>> simple problem,
>>>>> > >>>>> I would like to hear the community opinion. In our use case, a
>>>>> bit change
>>>>> > >>>>> in db_sync with filter by neutron key in external_ids would be
>>>>> enough.
>>>>> > >>>>>
>>>>> > >>>>> Regards,
>>>>> > >>>>> Roberto
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> [1]
>>>>> > >>>>>
>>>>> https://lists.openstack.org/pipermail/openstack-discuss/2023-March/032624.html
>>>>> > >>>>> [2] https://etherpad.opendev.org/p/neutron-bobcat-ptg
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> Additional logs:
>>>>> > >>>>>
>>>>> > >>>>> OpenStack 1
>>>>> > >>>>>
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>>>>> ovn-nbctl
>>>>> > >>>>> lr-route-list 6b776115-746a-4c59-aa73-6674c70b3498
>>>>> > >>>>> IPv4 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>               20.0.1.0/24             169.254.200.2 dst-ip
>>>>> (learned)
>>>>> > >>>>>               20.0.2.0/24             169.254.200.3 dst-ip
>>>>> (learned)
>>>>> > >>>>>                 0.0.0.0/0             200.200.200.1 dst-ip
>>>>> > >>>>>
>>>>> > >>>>> IPv6 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>                      ::/0     fc00:ca5a:ca5a:8000:: dst-ip
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>>>>> ovn-nbctl
>>>>> > >>>>> lr-route-list 23d4552a-62c4-40e1-8bae-d06af3489c07
>>>>> > >>>>> IPv4 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>               10.0.1.0/24             169.254.100.2 dst-ip
>>>>> (learned)
>>>>> > >>>>>               10.0.2.0/24             169.254.100.3 dst-ip
>>>>> (learned)
>>>>> > >>>>>                 0.0.0.0/0             200.200.200.1 dst-ip
>>>>> > >>>>>
>>>>> > >>>>> IPv6 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>                      ::/0     fc00:ca5a:ca5a:8000:: dst-ip
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f931b37c:~#
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> OpenStack 2
>>>>> > >>>>>
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-30f7e935:~#
>>>>> ovn-nbctl
>>>>> > >>>>> lr-route-list dc1e5008-adb9-451e-8b71-09388f3680bc
>>>>> > >>>>> IPv4 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>               20.0.0.0/24             169.254.200.1 dst-ip
>>>>> (learned)
>>>>> > >>>>>               20.0.2.0/24             169.254.200.3 dst-ip
>>>>> (learned)
>>>>> > >>>>>                 0.0.0.0/0             200.200.200.1 dst-ip
>>>>> > >>>>>
>>>>> > >>>>> IPv6 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>                      ::/0     fc00:ca5a:ca5a:8000:: dst-ip
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-30f7e935:~#
>>>>> ovn-nbctl
>>>>> > >>>>> lr-route-list ce45f681-6454-43fe-974f-81344bb8113a
>>>>> > >>>>> IPv4 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>               10.0.0.0/24             169.254.100.1 dst-ip
>>>>> (learned)
>>>>> > >>>>>               10.0.2.0/24             169.254.100.3 dst-ip
>>>>> (learned)
>>>>> > >>>>>                 0.0.0.0/0             200.200.200.1 dst-ip
>>>>> > >>>>>
>>>>> > >>>>> IPv6 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>                      ::/0     fc00:ca5a:ca5a:8000:: dst-ip
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> OpenStack 3
>>>>> > >>>>>
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f237db97:~#
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f237db97:~#
>>>>> ovn-nbctl
>>>>> > >>>>> lr-route-list  cfa259d6-311f-4409-bcf2-79a929835cb3
>>>>> > >>>>> IPv4 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>               20.0.0.0/24             169.254.200.1 dst-ip
>>>>> (learned)
>>>>> > >>>>>               20.0.1.0/24             169.254.200.2 dst-ip
>>>>> (learned)
>>>>> > >>>>>                 0.0.0.0/0             200.200.200.1 dst-ip
>>>>> > >>>>>
>>>>> > >>>>> IPv6 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>                      ::/0     fc00:ca5a:ca5a:8000:: dst-ip
>>>>> > >>>>> root at os-infra-1-neutron-ovn-northd-container-f237db97:~#
>>>>> ovn-nbctl
>>>>> > >>>>> lr-route-list  c5a4dcd8-b9a6-4397-a7cf-88bc1e01b0b0
>>>>> > >>>>> IPv4 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>               10.0.0.0/24             169.254.100.1 dst-ip
>>>>> (learned)
>>>>> > >>>>>               10.0.1.0/24             169.254.100.2 dst-ip
>>>>> (learned)
>>>>> > >>>>>                 0.0.0.0/0             200.200.200.1 dst-ip
>>>>> > >>>>>
>>>>> > >>>>> IPv6 Routes
>>>>> > >>>>> Route Table <main>:
>>>>> > >>>>>                      ::/0     fc00:ca5a:ca5a:8000:: dst-ip
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> OVN-IC Global database
>>>>> > >>>>>
>>>>> > >>>>> root at ovn-global-db1:~# ovn-ic-sbctl show
>>>>> > >>>>> availability-zone osp1
>>>>> > >>>>>     gateway 832b6c0d-13ce-4600-ab37-78516d8ec4c5
>>>>> > >>>>>         hostname: osp1-gwnode1
>>>>> > >>>>>         type: geneve
>>>>> > >>>>>             ip: 192.168.200.28
>>>>> > >>>>>         port admin-rt1-tenant1
>>>>> > >>>>>             transit switch: admin-tenant1
>>>>> > >>>>>             address: ["aa:aa:aa:aa:bb:01 169.254.100.1/24
>>>>> fe80::1/64"]
>>>>> > >>>>>         port admin-rt1-tenant1_1
>>>>> > >>>>>             transit switch: admin-tenant1_1
>>>>> > >>>>>             address: ["aa:aa:aa:aa:dd:01 169.254.200.1/24"]
>>>>> > >>>>> availability-zone osp2
>>>>> > >>>>>     gateway 17ffabdf-cf47-41ab-9539-d326c13c4ca8
>>>>> > >>>>>         hostname: osp2-gwnode1
>>>>> > >>>>>         type: geneve
>>>>> > >>>>>             ip: 192.168.200.128
>>>>> > >>>>>         port admin-rt2-tenant1
>>>>> > >>>>>             transit switch: admin-tenant1
>>>>> > >>>>>             address: ["aa:aa:aa:aa:bb:02 169.254.100.2/24
>>>>> fe80::2/64"]
>>>>> > >>>>>         port admin-rt2-tenant1_1
>>>>> > >>>>>             transit switch: admin-tenant1_1
>>>>> > >>>>>             address: ["aa:aa:aa:aa:dd:02 169.254.200.2/24"]
>>>>> > >>>>> availability-zone osp3
>>>>> > >>>>>     gateway 97595af9-7896-40d0-a883-beadbff1aa5b
>>>>> > >>>>>         hostname: osp3-gwnode1
>>>>> > >>>>>         type: geneve
>>>>> > >>>>>             ip: 192.168.200.228
>>>>> > >>>>>         port admin-rt3-tenant1
>>>>> > >>>>>             transit switch: admin-tenant1
>>>>> > >>>>>             address: ["aa:aa:aa:aa:aa:03 169.254.100.3/24
>>>>> fe80::3/64"]
>>>>> > >>>>>         port admin-rt3-tenant1_1
>>>>> > >>>>>             transit switch: admin-tenant1_1
>>>>> > >>>>>             address: ["aa:aa:aa:aa:dd:03 169.254.200.3/24"]
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> *‘Esta mensagem é direcionada apenas para os endereços
>>>>> constantes no
>>>>> > >>>>> cabeçalho inicial. Se você não está listado nos endereços
>>>>> constantes no
>>>>> > >>>>> cabeçalho, pedimos-lhe que desconsidere completamente o
>>>>> conteúdo dessa
>>>>> > >>>>> mensagem e cuja cópia, encaminhamento e/ou execução das ações
>>>>> citadas estão
>>>>> > >>>>> imediatamente anuladas e proibidas’.*
>>>>> > >>>>>
>>>>> > >>>>>  *‘Apesar do Magazine Luiza tomar todas as precauções
>>>>> razoáveis para
>>>>> > >>>>> assegurar que nenhum vírus esteja presente nesse e-mail, a
>>>>> empresa não
>>>>> > >>>>> poderá aceitar a responsabilidade por quaisquer perdas ou
>>>>> danos causados
>>>>> > >>>>> por esse e-mail ou por seus anexos’.*
>>>>> > >>>>>
>>>>> > >>>>
>>>>> > >>
>>>>> > >> *‘Esta mensagem é direcionada apenas para os endereços constantes
>>>>> no
>>>>> > >> cabeçalho inicial. Se você não está listado nos endereços
>>>>> constantes no
>>>>> > >> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo
>>>>> dessa
>>>>> > >> mensagem e cuja cópia, encaminhamento e/ou execução das ações
>>>>> citadas estão
>>>>> > >> imediatamente anuladas e proibidas’.*
>>>>> > >>
>>>>> > >>  *‘Apesar do Magazine Luiza tomar todas as precauções razoáveis
>>>>> para
>>>>> > >> assegurar que nenhum vírus esteja presente nesse e-mail, a
>>>>> empresa não
>>>>> > >> poderá aceitar a responsabilidade por quaisquer perdas ou danos
>>>>> causados
>>>>> > >> por esse e-mail ou por seus anexos’.*
>>>>> > >>
>>>>> > >
>>>>> >
>>>>> > --
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > _‘Esta mensagem é direcionada apenas para os endereços constantes no
>>>>> > cabeçalho inicial. Se você não está listado nos endereços constantes
>>>>> no
>>>>> > cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo
>>>>> dessa
>>>>> > mensagem e cuja cópia, encaminhamento e/ou execução das ações
>>>>> citadas estão
>>>>> > imediatamente anuladas e proibidas’._
>>>>> >
>>>>> >
>>>>> > * **‘Apesar do Magazine Luiza tomar
>>>>> > todas as precauções razoáveis para assegurar que nenhum vírus esteja
>>>>> > presente nesse e-mail, a empresa não poderá aceitar a
>>>>> responsabilidade por
>>>>> > quaisquer perdas ou danos causados por esse e-mail ou por seus
>>>>> anexos’.*
>>>>> >
>>>>> >
>>>>> >
>>>>> Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur
>>>>> für die Verwertung durch den vorgesehenen Empfänger bestimmt.
>>>>> Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den
>>>>> Absender bitte unverzüglich in Kenntnis und löschen diese E Mail.
>>>>>
>>>>> Hinweise zum Datenschutz finden Sie hier<
>>>>> https://www.datenschutz.schwarz>.
>>>>>
>>>>>
>>>>> This e-mail may contain confidential content and is intended only for
>>>>> the specified recipient/s.
>>>>> If you are not the intended recipient, please inform the sender
>>>>> immediately and delete this e-mail.
>>>>>
>>>>> Information on data protection can be found here<
>>>>> https://www.datenschutz.schwarz>.
>>>>>
>>>>>
>>>
>>> *‘Esta mensagem é direcionada apenas para os endereços constantes no
>>> cabeçalho inicial. Se você não está listado nos endereços constantes no
>>> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
>>> mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
>>> imediatamente anuladas e proibidas’.*
>>>
>>>  *‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para
>>> assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não
>>> poderá aceitar a responsabilidade por quaisquer perdas ou danos causados
>>> por esse e-mail ou por seus anexos’.*
>>>
>>
>
> *‘Esta mensagem é direcionada apenas para os endereços constantes no
> cabeçalho inicial. Se você não está listado nos endereços constantes no
> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
> mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
> imediatamente anuladas e proibidas’.*
>
>  *‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para
> assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não
> poderá aceitar a responsabilidade por quaisquer perdas ou danos causados
> por esse e-mail ou por seus anexos’.*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230717/db1ad968/attachment-0001.htm>