Open Stack

Hey Luis,

Thanks for your mail.

We have now prepared a first draft. In addition to what Ihtisham already wrote, we need the following options:
- to run multiple agents per host (container) because we need more than one BGP session, alternatively there would be the option to do it via frr
- we need to be able to filter which networks we announce where, for this we used the address scope from Openstack in the past

To make this possible we have built it in so that it runs through the address scope. To make the address scope usable in ovn-bgp-agent we also patched Neutron so that the address scope is part of router_gateway and router_interface patch-ports. We also announce the networks behind the routers instead of the VM IPs directly.

We added a new driver for this because we don't really need anything from the previous implementation at the moment and we were missing the similarities between the two. Possibly someone has an idea how we could merge this?

It would be nice if you could have a first look at it.

There is still some work to do like adding tests and making Zuul happy but I think it is useful to discuss this with you as early as possible.

You can find the merge request here: https://review.opendev.org/c/x/ovn-bgp-agent/+/861581

Best regards,
Luca Czesla

From: Luis Tomas Bolivar <ltomasbo at redhat.com>
Sent: Monday, October 3, 2022 7:45 AM
To: Ihtisham ul Haq <Ihtisham.ul_Haq at mail.schwarz>
Cc: Daniel Alvarez <dalvarez at redhat.com>; Luca Czesla <Luca.Czesla at mail.schwarz>; Max André Lamprecht <Max.Lamprecht at mail.schwarz>; openstack-discuss <openstack-discuss at lists.openstack.org>
Subject: Re: [ovn][neutron] RE: OVN BGP Agent query



On Fri, Sep 30, 2022 at 6:20 PM Ihtisham ul Haq <Ihtisham.ul_Haq at mail.schwarz<mailto:Ihtisham.ul_Haq at mail.schwarz>> wrote:
Hi Luis and Daniel,

Please see inline response.

> From: Daniel Alvarez Sanchez <dalvarez at redhat.com<mailto:dalvarez at redhat.com>>
> Sent: 29 September 2022 11:37
> Subject: Re: OVN BGP Agent query
>
> Hi Ihtisham and Luis,
>
> On Thu, Sep 29, 2022 at 7:42 AM Luis Tomas Bolivar <ltomasbo at redhat.com<mailto:ltomasbo at redhat.com>> wrote:
> > Some comments and questions inline
> >
> > On Tue, Sep 27, 2022 at 1:39 PM Ihtisham ul haq <ihtisham.uh at hotmail.com<mailto:ihtisham.uh at hotmail.com>> wrote:
> > > Hi Luis,
> > >
> > > Thanks for your work on the OVN BGP Agent. We are planning
> > > to use it in our OVN deployment, but have a question regarding it.
> >
> > Great to hear! Can you share a bit more info about this environment? like
> > openstack version, target workload, etc.

We plan to use this with Yoga version. Our workload consist of enterprise users
with VMs running on Openstack and connected to their enterprise network via
transfer network(to which the customer neutron router is attached to).
And we also have public workload but with the ovn-bgp we only want to
we want to advertise the former.

> >
> > > The way our current setup with ML2/OVS works is that our customer VM IP routes
> > > are announced via the router IP(of the that customer) to the leaf switch instead of
> > > the IP of the host where the neutron BGP agent runs. And then even if the
> > > router fails over, the IP of the router stays the same and thus the BGP route
> > > doesn't need to be updated.
> >
> > Is this with Neutron Dynamic Routing? When you say Router IP, do you mean the virtual neutron router and its IP associated with the provider network? What type of IPs are you announcing with BGP? IPs on provider network or on tenant networks (or both)?

Yes, that's with Neutron DR agent, and I meant virtual neutron router with
IP from the provider network. We announce IPs of our tenant network via the
virtual routers external address.

> > If the router fails over, the route needs to be updated, doesn't it? Same IP, but exposed in the new location of the router?

Correct.

> The route to the tenant network doesn't change, ie.
> 192.168.0.0 via 172.24.4.100  (this route remains the same regardless of where 172.24.4.100 is).
> If there's L2 in the 172.24.4.0 network, the new location of 172.24.4.100 will be learnt via GARP announcement. In our case, this won't happen as we don't have L2 so we expose directly connected routes to overcome this "limitation".

Right, in our case we have a stretched L2 transfer network(mentioned above)
to which our gateway nodes and customer routers are connected to, so we can
advertise the IPs from the tenant network via the virtual router external IP
and thus the location of the router isn't relevant in case of failover as its
address will be relearned.

> In the case of Neutron Dynamic Routing, there's no assumption that everything is L3 so GARPs are needed to learn the new location.
>
> > > We see that the routes are announced by the ovn-bgp-agent via the host IP(GTW) in our
> > > switch peers. If that's the case then how do you make sure that during failover
> > > of a router, the BGP routes gets updated with the new host IP(where the router
> > > failed over to)?
> >
> > The local FRR running at each node is in charge of exposing the IPs. For the IPs on the provider network, the traffic is directly exposed where the VMs are, without having to go through the virtual router, so a router failover won't change the route.
> > In the case of VMs on tenant networks, the traffic is exposed on the node where the virtual router gateway port is associated (I suppose this is what you refer to with router IP). In the case of a failover the agent is in charge of making FRR to withdraw the exposed routes on the old node, and re-advertise them on the new router IP location
> >
> > > Can we accomplish the same route advertisement as our ML2/OVS setup, using the ovn-bgp-agent?
>
> I think this is technically possible, and perhaps you want to contribute that functionality or even help integrating the agent as a driver of Neutron Dynamic Routing?

Sounds good, our plan currently is to add this to the ovn-bgp-agent,
so we can announce our tenant routes via virtual routers external address on
a stretched L2 network, to make it work with our use case.

Great to hear!!

Just to make it clear, the ovn-bgp-agent current solution is to expose the tenant VM IPs through the host that has the OVN router gateway port, so for example, if the VM IP (10.0.0.5) is connected to the neutron virtual router, which in turns is connected to your provider network (your transfer network) with IP 172.24.4.10, and hosted in a physical server with IP 192.168.100.100, the route will be exposed as:
- 10.0.0.5 nexthop 192.168.100.100
- 172.24.4.10 nexthop 192.168.100.100

As we are using FRR config "redistributed connected". As the traffic to the tenant networks needs to be injected into the OVN overlay through the gateway node hosting that ovn virtual router gateway port (cr-lrp), would it be ok if, besides those route we also advertise?
- 10.0.0.5 nexthop 172.24.4.10

Cheers,
Luis








--
Ihtisham ul Haq
Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie hier<https://www.datenschutz.schwarz<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.datenschutz.schwarz%2F&data=05%7C01%7C%7C0635d1ca0f4d4281fff108daa502667b%7Cd04f47175a6e4b98b3f96918e0385f4c%7C0%7C0%7C638003726972382016%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=sFol3dUCKOU6UiSMk4Y2%2BLwXyZRoe83LKbANBq%2Bi7qg%3D&reserved=0>>.


--
LUIS TOMÁS BOLÍVAR
Principal Software Engineer
Red Hat
Madrid, Spain
ltomasbo at redhat.com<mailto:ltomasbo at redhat.com>


Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie hier<https://www.datenschutz.schwarz>.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20221017/f2ef425c/attachment-0001.htm>