Open Stack

Hello Luca!

Awesome patch! See some comments below, though I'll also add some more
details on the gerrit side

On Mon, Oct 17, 2022 at 11:12 AM Luca Czesla <Luca.Czesla at mail.schwarz>
wrote:

> Hey Luis,
>
>
>
> Thanks for your mail.
>
>
>
> We have now prepared a first draft. In addition to what Ihtisham already
> wrote, we need the following options:
>
> - to run multiple agents per host (container) because we need more than
> one BGP session, alternatively there would be the option to do it via frr
>

Nice, this could actually be splitted from the patch into a different one,
as it is kind of independent of the new driver, and can be used for other
use cases (and merge it almost right away)


> - we need to be able to filter which networks we announce where, for this
> we used the address scope from Openstack in the past
>

>
> To make this possible we have built it in so that it runs through the
> address scope. To make the address scope usable in ovn-bgp-agent we also
> patched Neutron so that the address scope is part of router_gateway and
> router_interface patch-ports. We also announce the networks behind the
> routers instead of the VM IPs directly.
>

We actually play with something similar to be able to have a simple/initial
API here: https://review.opendev.org/c/openstack/neutron/+/797087 (idea was
to use port tags instead)

Do you have a link to the modifications to enable this in the neutron side?


>
>
> We added a new driver for this because we don't really need anything from
> the previous implementation at the moment and we were missing the
> similarities between the two. Possibly someone has an idea how we could
> merge this?
>

Is the new expose_router_port/withdraw_router_port similar to the previous
expose_subnet/withdraw_subnet? Perhaps we just need a different
implementation of those?

I was actually playing (after initial discussion with Ihtisha) for options
to not only expose VM_IP via HOST_IP, but doing it VM_IP though
OVN_GATEWAY_PORT_IP, and then GATEWAY_PORT_IP via HOST_IP. I was playing
with "redistribute static" instead of "redistributed kernel" + IP routes (I
think I like your approach better).

My idea was to have the existing driver with an option to state if the env
is pure L3 (the current support) or L2 (where you work is needed). That is
another option to explore, but open to having completely different drivers,
as it also makes a lot of sense.

>
>
> It would be nice if you could have a first look at it.
>
>
>
> There is still some work to do like adding tests and making Zuul happy but
> I think it is useful to discuss this with you as early as possible.
>

Yeah, no problem, we'll make zull happy after the initial design/idea is
discussed. Don't need to waste time on that at this moment

>
>
> You can find the merge request here:
> https://review.opendev.org/c/x/ovn-bgp-agent/+/861581
>

Thanks! I'll leave some more detailed review in there

There is one more general question on that patch. Isn't that exposing the
routes in all the nodes where the agent is running instead of where the
actual ovn router gateway port is attached? Don't you need to check is the
node where you can actually inject the traffic to OVN overlay? Or you are
assuming everything is L2 at that point, and it does not matter everyone
expose the route as at the end of the day, only the node with the ovn
gateway router port will reply to ARPs


>
> Best regards,
>
> Luca Czesla
>
>
>
> *From:* Luis Tomas Bolivar <ltomasbo at redhat.com>
> *Sent:* Monday, October 3, 2022 7:45 AM
> *To:* Ihtisham ul Haq <Ihtisham.ul_Haq at mail.schwarz>
> *Cc:* Daniel Alvarez <dalvarez at redhat.com>; Luca Czesla
> <Luca.Czesla at mail.schwarz>; Max André Lamprecht
> <Max.Lamprecht at mail.schwarz>; openstack-discuss <
> openstack-discuss at lists.openstack.org>
> *Subject:* Re: [ovn][neutron] RE: OVN BGP Agent query
>
>
>
>
>
>
>
> On Fri, Sep 30, 2022 at 6:20 PM Ihtisham ul Haq <
> Ihtisham.ul_Haq at mail.schwarz> wrote:
>
> Hi Luis and Daniel,
>
> Please see inline response.
>
> > From: Daniel Alvarez Sanchez <dalvarez at redhat.com>
> > Sent: 29 September 2022 11:37
> > Subject: Re: OVN BGP Agent query
> >
> > Hi Ihtisham and Luis,
> >
> > On Thu, Sep 29, 2022 at 7:42 AM Luis Tomas Bolivar <ltomasbo at redhat.com>
> wrote:
> > > Some comments and questions inline
> > >
> > > On Tue, Sep 27, 2022 at 1:39 PM Ihtisham ul haq <
> ihtisham.uh at hotmail.com> wrote:
> > > > Hi Luis,
> > > >
> > > > Thanks for your work on the OVN BGP Agent. We are planning
> > > > to use it in our OVN deployment, but have a question regarding it.
> > >
> > > Great to hear! Can you share a bit more info about this environment?
> like
> > > openstack version, target workload, etc.
>
> We plan to use this with Yoga version. Our workload consist of enterprise
> users
> with VMs running on Openstack and connected to their enterprise network via
> transfer network(to which the customer neutron router is attached to).
> And we also have public workload but with the ovn-bgp we only want to
> we want to advertise the former.
>
> > >
> > > > The way our current setup with ML2/OVS works is that our customer VM
> IP routes
> > > > are announced via the router IP(of the that customer) to the leaf
> switch instead of
> > > > the IP of the host where the neutron BGP agent runs. And then even
> if the
> > > > router fails over, the IP of the router stays the same and thus the
> BGP route
> > > > doesn't need to be updated.
> > >
> > > Is this with Neutron Dynamic Routing? When you say Router IP, do you
> mean the virtual neutron router and its IP associated with the provider
> network? What type of IPs are you announcing with BGP? IPs on provider
> network or on tenant networks (or both)?
>
> Yes, that's with Neutron DR agent, and I meant virtual neutron router with
> IP from the provider network. We announce IPs of our tenant network via the
> virtual routers external address.
>
> > > If the router fails over, the route needs to be updated, doesn't it?
> Same IP, but exposed in the new location of the router?
>
> Correct.
>
> > The route to the tenant network doesn't change, ie.
> > 192.168.0.0 via 172.24.4.100  (this route remains the same regardless of
> where 172.24.4.100 is).
> > If there's L2 in the 172.24.4.0 network, the new location of
> 172.24.4.100 will be learnt via GARP announcement. In our case, this won't
> happen as we don't have L2 so we expose directly connected routes to
> overcome this "limitation".
>
> Right, in our case we have a stretched L2 transfer network(mentioned above)
> to which our gateway nodes and customer routers are connected to, so we can
> advertise the IPs from the tenant network via the virtual router external
> IP
> and thus the location of the router isn't relevant in case of failover as
> its
> address will be relearned.
>
> > In the case of Neutron Dynamic Routing, there's no assumption that
> everything is L3 so GARPs are needed to learn the new location.
> >
> > > > We see that the routes are announced by the ovn-bgp-agent via the
> host IP(GTW) in our
> > > > switch peers. If that's the case then how do you make sure that
> during failover
> > > > of a router, the BGP routes gets updated with the new host IP(where
> the router
> > > > failed over to)?
> > >
> > > The local FRR running at each node is in charge of exposing the IPs.
> For the IPs on the provider network, the traffic is directly exposed where
> the VMs are, without having to go through the virtual router, so a router
> failover won't change the route.
> > > In the case of VMs on tenant networks, the traffic is exposed on the
> node where the virtual router gateway port is associated (I suppose this is
> what you refer to with router IP). In the case of a failover the agent is
> in charge of making FRR to withdraw the exposed routes on the old node, and
> re-advertise them on the new router IP location
> > >
> > > > Can we accomplish the same route advertisement as our ML2/OVS setup,
> using the ovn-bgp-agent?
> >
> > I think this is technically possible, and perhaps you want to contribute
> that functionality or even help integrating the agent as a driver of
> Neutron Dynamic Routing?
>
> Sounds good, our plan currently is to add this to the ovn-bgp-agent,
> so we can announce our tenant routes via virtual routers external address
> on
> a stretched L2 network, to make it work with our use case.
>
>
>
> Great to hear!!
>
>
>
> Just to make it clear, the ovn-bgp-agent current solution is to expose the
> tenant VM IPs through the host that has the OVN router gateway port, so for
> example, if the VM IP (10.0.0.5) is connected to the neutron virtual
> router, which in turns is connected to your provider network (your transfer
> network) with IP 172.24.4.10, and hosted in a physical server with IP
> 192.168.100.100, the route will be exposed as:
>
> - 10.0.0.5 nexthop 192.168.100.100
>
> - 172.24.4.10 nexthop 192.168.100.100
>
>
>
> As we are using FRR config "redistributed connected". As the traffic to
> the tenant networks needs to be injected into the OVN overlay through the
> gateway node hosting that ovn virtual router gateway port (cr-lrp), would
> it be ok if, besides those route we also advertise?
>
> - 10.0.0.5 nexthop 172.24.4.10
>
>
>
> Cheers,
>
> Luis
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Ihtisham ul Haq
> Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für
> die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht
> der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich
> in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie
> hier<https://www.datenschutz.schwarz
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.datenschutz.schwarz%2F&data=05%7C01%7C%7C0635d1ca0f4d4281fff108daa502667b%7Cd04f47175a6e4b98b3f96918e0385f4c%7C0%7C0%7C638003726972382016%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=sFol3dUCKOU6UiSMk4Y2%2BLwXyZRoe83LKbANBq%2Bi7qg%3D&reserved=0>
> >.
>
>
>
> --
>
> LUIS TOMÁS BOLÍVAR
> Principal Software Engineer
> Red Hat
> Madrid, Spain
> ltomasbo at redhat.com
>
>
> Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für
> die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht
> der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich
> in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie
> hier <https://www.datenschutz.schwarz>.
>


-- 
LUIS TOMÁS BOLÍVAR
Principal Software Engineer
Red Hat
Madrid, Spain
ltomasbo at redhat.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20221018/8f80a9d5/attachment-0001.htm>