Open Stack

Hey Luis,

thanks a lot for the great feedback.

See inline reply please.

Best regards,
Luca Czesla


From: Luis Tomas Bolivar <ltomasbo at redhat.com>
Sent: Tuesday, October 18, 2022 9:54 AM
To: Luca Czesla <Luca.Czesla at mail.schwarz>
Cc: Daniel Alvarez <dalvarez at redhat.com>; Max André Lamprecht <Max.Lamprecht at mail.schwarz>; openstack-discuss <openstack-discuss at lists.openstack.org>; Ihtisham ul Haq <Ihtisham.ul_Haq at mail.schwarz>
Subject: Re: [ovn][neutron] RE: OVN BGP Agent query

Hello Luca!

Awesome patch! See some comments below, though I'll also add some more details on the gerrit side

On Mon, Oct 17, 2022 at 11:12 AM Luca Czesla <Luca.Czesla at mail.schwarz<mailto:Luca.Czesla at mail.schwarz>> wrote:
Hey Luis,

Thanks for your mail.

We have now prepared a first draft. In addition to what Ihtisham already wrote, we need the following options:
- to run multiple agents per host (container) because we need more than one BGP session, alternatively there would be the option to do it via frr

Nice, this could actually be splitted from the patch into a different one, as it is kind of independent of the new driver, and can be used for other use cases (and merge it almost right away)


Good idea. I opened a separate merge request here: https://review.opendev.org/c/x/ovn-bgp-agent/+/861749

- we need to be able to filter which networks we announce where, for this we used the address scope from Openstack in the past

To make this possible we have built it in so that it runs through the address scope. To make the address scope usable in ovn-bgp-agent we also patched Neutron so that the address scope is part of router_gateway and router_interface patch-ports. We also announce the networks behind the routers instead of the VM IPs directly.

We actually play with something similar to be able to have a simple/initial API here: https://review.opendev.org/c/openstack/neutron/+/797087<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.opendev.org%2Fc%2Fopenstack%2Fneutron%2F%2B%2F797087&data=05%7C01%7C%7C1ad6d332cbd94ebf95e408dab0ddf376%7Cd04f47175a6e4b98b3f96918e0385f4c%7C0%7C0%7C638016764570286277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BwwlZI48MmY4fp5mSCaCDY7xYgN3CbmjaVF5QyfxWtg%3D&reserved=0> (idea was to use port tags instead)

Do you have a link to the modifications to enable this in the neutron side?


I just opened the MR at Neutron. It's pretty much what you do with the port_tags as well. You can find it here: https://review.opendev.org/c/openstack/neutron/+/861719
Is there an advantage of using "port_tags" at this point, or is that simply a catch-all term for metadata? I am fine with both solutions and have no preferences there.


We added a new driver for this because we don't really need anything from the previous implementation at the moment and we were missing the similarities between the two. Possibly someone has an idea how we could merge this?

Is the new expose_router_port/withdraw_router_port similar to the previous expose_subnet/withdraw_subnet? Perhaps we just need a different implementation of those?


I am not sure if it is the right way. I had problems for example with the withdraw_subnet that I could not get the corresponding router port from the lrp port because it was already deleted, so I was missing the reference to the address scope. So if you have a better way to get to the target, I will gladly accept it.

The function is similar only that I didn't want to use the lrp ports for the reason above and we need the update event since initial row.mac is unknown and will only be set to router with the next revision number. I am not sure how we could merge this but I am of course open for ideas.
Another idea would be to have another separate watcher that implements only these two events. Maybe that would be the cleanest way?


I was actually playing (after initial discussion with Ihtisha) for options to not only expose VM_IP via HOST_IP, but doing it VM_IP though OVN_GATEWAY_PORT_IP, and then GATEWAY_PORT_IP via HOST_IP. I was playing with "redistribute static" instead of "redistributed kernel" + IP routes (I think I like your approach better).

My idea was to have the existing driver with an option to state if the env is pure L3 (the current support) or L2 (where you work is needed). That is another option to explore, but open to having completely different drivers, as it also makes a lot of sense.

It would be nice if you could have a first look at it.

There is still some work to do like adding tests and making Zuul happy but I think it is useful to discuss this with you as early as possible.

Yeah, no problem, we'll make zull happy after the initial design/idea is discussed. Don't need to waste time on that at this moment

You can find the merge request here: https://review.opendev.org/c/x/ovn-bgp-agent/+/861581<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.opendev.org%2Fc%2Fx%2Fovn-bgp-agent%2F%2B%2F861581&data=05%7C01%7C%7C1ad6d332cbd94ebf95e408dab0ddf376%7Cd04f47175a6e4b98b3f96918e0385f4c%7C0%7C0%7C638016764570286277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=M7AF9ux%2FdEg6uxQ9TlJaEkphNV3KjOVLOJO%2F3Qd1P%2F8%3D&reserved=0>

Thanks! I'll leave some more detailed review in there


Thank you very much. I will try to work on it tomorrow.


There is one more general question on that patch. Isn't that exposing the routes in all the nodes where the agent is running instead of where the actual ovn router gateway port is attached? Don't you need to check is the node where you can actually inject the traffic to OVN overlay? Or you are assuming everything is L2 at that point, and it does not matter everyone expose the route as at the end of the day, only the node with the ovn gateway router port will reply to ARPs


Yes exactly, we assume that the respective router IPs/Networks are properly filtered via the address scope and that the BGP peers are in the same L2 network as the provider network of the routers. The patch is really only about announcing the tenant networks via the correct router IP. We are not interested in where this IP is located, this task can be done by ARP in the respective L2 network. So everything can be routed locally in the L2 via ARP lookups and every BGP agent that has a foot in the network can announce the networks identically. So in the end we have N times the same route which only differ from whom it was received. Announcing the complete networks makes sense for us as we have less BGP updates and the routers can drop the traffic. The advantage of this is that L3 failover is instant, since no new BGP route needs to be announced. The switching of the router IP to another gateway/chassis is then done by OVN using GARP.



Best regards,
Luca Czesla

From: Luis Tomas Bolivar <ltomasbo at redhat.com<mailto:ltomasbo at redhat.com>>
Sent: Monday, October 3, 2022 7:45 AM
To: Ihtisham ul Haq <Ihtisham.ul_Haq at mail.schwarz<mailto:Ihtisham.ul_Haq at mail.schwarz>>
Cc: Daniel Alvarez <dalvarez at redhat.com<mailto:dalvarez at redhat.com>>; Luca Czesla <Luca.Czesla at mail.schwarz<mailto:Luca.Czesla at mail.schwarz>>; Max André Lamprecht <Max.Lamprecht at mail.schwarz<mailto:Max.Lamprecht at mail.schwarz>>; openstack-discuss <openstack-discuss at lists.openstack.org<mailto:openstack-discuss at lists.openstack.org>>
Subject: Re: [ovn][neutron] RE: OVN BGP Agent query



On Fri, Sep 30, 2022 at 6:20 PM Ihtisham ul Haq <Ihtisham.ul_Haq at mail.schwarz<mailto:Ihtisham.ul_Haq at mail.schwarz>> wrote:
Hi Luis and Daniel,

Please see inline response.

> From: Daniel Alvarez Sanchez <dalvarez at redhat.com<mailto:dalvarez at redhat.com>>
> Sent: 29 September 2022 11:37
> Subject: Re: OVN BGP Agent query
>
> Hi Ihtisham and Luis,
>
> On Thu, Sep 29, 2022 at 7:42 AM Luis Tomas Bolivar <ltomasbo at redhat.com<mailto:ltomasbo at redhat.com>> wrote:
> > Some comments and questions inline
> >
> > On Tue, Sep 27, 2022 at 1:39 PM Ihtisham ul haq <ihtisham.uh at hotmail.com<mailto:ihtisham.uh at hotmail.com>> wrote:
> > > Hi Luis,
> > >
> > > Thanks for your work on the OVN BGP Agent. We are planning
> > > to use it in our OVN deployment, but have a question regarding it.
> >
> > Great to hear! Can you share a bit more info about this environment? like
> > openstack version, target workload, etc.

We plan to use this with Yoga version. Our workload consist of enterprise users
with VMs running on Openstack and connected to their enterprise network via
transfer network(to which the customer neutron router is attached to).
And we also have public workload but with the ovn-bgp we only want to
we want to advertise the former.

> >
> > > The way our current setup with ML2/OVS works is that our customer VM IP routes
> > > are announced via the router IP(of the that customer) to the leaf switch instead of
> > > the IP of the host where the neutron BGP agent runs. And then even if the
> > > router fails over, the IP of the router stays the same and thus the BGP route
> > > doesn't need to be updated.
> >
> > Is this with Neutron Dynamic Routing? When you say Router IP, do you mean the virtual neutron router and its IP associated with the provider network? What type of IPs are you announcing with BGP? IPs on provider network or on tenant networks (or both)?

Yes, that's with Neutron DR agent, and I meant virtual neutron router with
IP from the provider network. We announce IPs of our tenant network via the
virtual routers external address.

> > If the router fails over, the route needs to be updated, doesn't it? Same IP, but exposed in the new location of the router?

Correct.

> The route to the tenant network doesn't change, ie.
> 192.168.0.0 via 172.24.4.100  (this route remains the same regardless of where 172.24.4.100 is).
> If there's L2 in the 172.24.4.0 network, the new location of 172.24.4.100 will be learnt via GARP announcement. In our case, this won't happen as we don't have L2 so we expose directly connected routes to overcome this "limitation".

Right, in our case we have a stretched L2 transfer network(mentioned above)
to which our gateway nodes and customer routers are connected to, so we can
advertise the IPs from the tenant network via the virtual router external IP
and thus the location of the router isn't relevant in case of failover as its
address will be relearned.

> In the case of Neutron Dynamic Routing, there's no assumption that everything is L3 so GARPs are needed to learn the new location.
>
> > > We see that the routes are announced by the ovn-bgp-agent via the host IP(GTW) in our
> > > switch peers. If that's the case then how do you make sure that during failover
> > > of a router, the BGP routes gets updated with the new host IP(where the router
> > > failed over to)?
> >
> > The local FRR running at each node is in charge of exposing the IPs. For the IPs on the provider network, the traffic is directly exposed where the VMs are, without having to go through the virtual router, so a router failover won't change the route.
> > In the case of VMs on tenant networks, the traffic is exposed on the node where the virtual router gateway port is associated (I suppose this is what you refer to with router IP). In the case of a failover the agent is in charge of making FRR to withdraw the exposed routes on the old node, and re-advertise them on the new router IP location
> >
> > > Can we accomplish the same route advertisement as our ML2/OVS setup, using the ovn-bgp-agent?
>
> I think this is technically possible, and perhaps you want to contribute that functionality or even help integrating the agent as a driver of Neutron Dynamic Routing?

Sounds good, our plan currently is to add this to the ovn-bgp-agent,
so we can announce our tenant routes via virtual routers external address on
a stretched L2 network, to make it work with our use case.

Great to hear!!

Just to make it clear, the ovn-bgp-agent current solution is to expose the tenant VM IPs through the host that has the OVN router gateway port, so for example, if the VM IP (10.0.0.5) is connected to the neutron virtual router, which in turns is connected to your provider network (your transfer network) with IP 172.24.4.10, and hosted in a physical server with IP 192.168.100.100, the route will be exposed as:
- 10.0.0.5 nexthop 192.168.100.100
- 172.24.4.10 nexthop 192.168.100.100

As we are using FRR config "redistributed connected". As the traffic to the tenant networks needs to be injected into the OVN overlay through the gateway node hosting that ovn virtual router gateway port (cr-lrp), would it be ok if, besides those route we also advertise?
- 10.0.0.5 nexthop 172.24.4.10

Cheers,
Luis








--
Ihtisham ul Haq
Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie hier<https://www.datenschutz.schwarz<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.datenschutz.schwarz%2F&data=05%7C01%7C%7C1ad6d332cbd94ebf95e408dab0ddf376%7Cd04f47175a6e4b98b3f96918e0385f4c%7C0%7C0%7C638016764570286277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2CQpWJ1F4e6lQ2gchNCAQxW0EFzapKk6vP3VS6Buch4%3D&reserved=0>>.


--
LUIS TOMÁS BOLÍVAR
Principal Software Engineer
Red Hat
Madrid, Spain
ltomasbo at redhat.com<mailto:ltomasbo at redhat.com>


Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie hier<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.datenschutz.schwarz%2F&data=05%7C01%7C%7C1ad6d332cbd94ebf95e408dab0ddf376%7Cd04f47175a6e4b98b3f96918e0385f4c%7C0%7C0%7C638016764570286277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2CQpWJ1F4e6lQ2gchNCAQxW0EFzapKk6vP3VS6Buch4%3D&reserved=0>.


--
LUIS TOMÁS BOLÍVAR
Principal Software Engineer
Red Hat
Madrid, Spain
ltomasbo at redhat.com<mailto:ltomasbo at redhat.com>



Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie hier<https://www.datenschutz.schwarz>.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20221018/4494b4eb/attachment-0001.htm>