Open Stack

You dont need a stretched provisioning network when you setup a DHCP relay
:)

IMO the L2 external network in Neutron is a major issue in OpenStack
scaling. I’d love to see a BGP support in OVN and OVN neutron plugin.


On Tue, 12 Oct 2021 at 21:28 Francois <rigault.francois at gmail.com> wrote:

> (yes we are using distributed fips) Well we don't want stretched
> VLANs. However... if we followed the doc we would end up with 3
> controllers in the same rack which would not be resilient. Since we
> have just 3 controllers plugged on specific, identified ports, we
> afford to have a stretched VLAN on these few ports only. For the
> provisioning network, I am taking a shortcut since this network should
> basically only be needed once in a while for stack upgrades and
> nothing interesting (like mac addresses moving) happens there. The
> data plane traffic, that needs scalability and resiliency, is not
> going through these VLANs. I think stretched VLANs on leaf spine
> networks are forbidden in general for these reasons (large L2
> networks? STP reducing the bandwidth? broadcast storm? larger failure
> domain? I don't know specifically, I would need help from a network
> engineer to explain the reason).
>
>
> https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/spine_leaf_networking/index
>
>
> On Tue, 12 Oct 2021 at 18:04, Tony Liu <tonyliu0592 at hotmail.com> wrote:
> >
> > I wonder, since there is already VXLAN based L2 across multiple racks
> > (which means you are not looking for pure L3 solution),
> > while keep tenant network multi-subnet on L3 for EW traffic,
> > why not have external network also on L2 and stretched on multiple racks
> > for NS traffic, assuming you are using distributed FIP?
> >
> > Thanks!
> > Tony
> > ________________________________________
> > From: Francois <rigault.francois at gmail.com>
> > Sent: October 12, 2021 07:03 AM
> > To: openstack-discuss
> > Subject: [neutron] OVN and dynamic routing
> >
> > Hello Neutron!
> > I am looking into running stacks with OVN on a leaf-spine network, and
> > have some floating IPs routed between racks.
> >
> > Basically each rack is assigned its own set of subnets.
> > Some VLANs are stretched across all racks: the provisioning VLAN used
> > by tripleo to deploy the stack, and the VLANs for the controllers API
> > IPs. However, each tenant subnet is local to a rack: for example each
> > OVN chassis has a ovn-encap-ip with an IP of the tenant subnet of its
> > own rack. Traffic between 2 racks is sent to a spine, and leaves and
> > spines run some eVPN-like thing: each pair of ToR is a vtep, traffic
> > is encapsulated as VXLAN, and routes between vteps are exchanged with
> > BGP.
> >
> > I am looking into supporting floating IPs in there: I expect floating
> > IPs to be able to move between racks, as such I am looking into
> > publishing the route for a FIP towards an hypervisor, through BGP.
> > Each fip is a /32 subnet with an hypervisor tenant's IP as next hop.
> >
> > It seems there are several ideas to achieve this (it was discussed
> > [before][1] in ovs conference)
> > - using [neutron-dynamic-routing][2] - that seems to have some gaps
> > for OVN. It uses os-ken to talk to switches and exchange routes
> > - using [OVN BGP agent][3] that uses FRR, it seems there is a related
> > [RFE][4] for integration in tripleo
> >
> > There is btw also a [BGPVPN][5] project (it does not match my usecase
> > as far as I tried to understand it) that also has some code that talks
> > BGP to switches, already integrated in tripleo.
> >
> > For my tests, I was able to use the neutron-dynamic-routing project
> > (almost) as documented, with a few changes:
> > - for traffic going from VMs to outside the stack, the hypervisor was
> > trying to resolve the "gateway of fIPs" with ARP request which does
> > not make any sense. I created a dummy port with the mac address of the
> > virtual router of the switches:
> > ```
> > $ openstack port list --mac-address 00:1c:73:00:00:11 -f yaml
> > - Fixed IP Addresses:
> > - ip_address: 10.64.254.1
> > subnet_id: 8f37
> > ID: 4028
> > MAC Address: 00:1c:73:00:00:11
> > Name: lagw
> > Status: DOWN
> > ```
> > this prevent the hypervisor to send ARP requests to a non existent
> gateway
> > - for traffic coming back, we start the neutron-bgp-dragent agent on
> > the controllers. We create the right bgp speaker, peers, etc.
> > - neutron-bgp-dragent seems to work primarily with ovs ml2 plugin, it
> > selects fips and join with ports owned by a "floatingip_agent_gateway"
> > which does not exist on OVN. We can define ourselves some ports so
> > that the dragent is able to find the tenant IP of a host:
> > ```
> > openstack port create --network provider --device-owner
> > network:floatingip_agent_gateway --host cpu35d.cloud --fixed-ip
> > ip-address=10.64.245.102 ag2
> > ```
> > - when creating a floating IP and assigning a port to it, Neutron
> > reads changes from OVN SB and fills the binding information into the
> > port:
> > ```
> > $ openstack port show -c binding_host_id `openstack floating ip show
> > 10.64.254.177 -f value -c port_id`
> > +-----------------+----------------------------------------+
> > | Field | Value |
> > +-----------------+----------------------------------------+
> > | binding_host_id | cpu35d.cloud |
> > +-----------------+----------------------------------------+
> > ```
> > this allows the dragent to publish the route for the fip
> > ```
> > $ openstack bgp speaker list advertised routes bgpspeaker
> > +------------------+---------------+
> > | Destination | Nexthop |
> > +------------------+---------------+
> > | 10.64.254.177/32 | 10.64.245.102 |
> > +------------------+---------------+
> > ```
> > - traffic reaches the hypervisor but (for reason I don't understand) I
> > had to add a rule
> > ```
> > $ ip rule
> > 0: from all lookup local
> > 32765: from all iif vlan1234 lookup ovn
> > 32766: from all lookup main
> > 32767: from all lookup default
> > $ ip route show table ovn
> > 10.64.254.177 dev vlan1234 scope link
> > ```
> > so that the traffic coming for the fip is not immediately discarded by
> > the hypervisor (it's not an ideal solution but it is a workaround that
> > makes my one fIP work!)
> >
> > So all in all it seems it would be possible to use the
> > neutron-dynamic-routing agent, with some minor modifications (eg: to
> > also publish the fip of the OVN L3 gateway router).
> >
> > I am wondering whether I have overlooked anything, and if such kind of
> > deployment (OVN + neutron dynamic routing or similar) is already in
> > use somewhere. Does it make sense to have a RFE for better integration
> > between OVN and neutron-dynamic-routing?
> >
> > Thanks
> > Francois
> >
> >
> >
> >
> > [1]:
> https://www.openvswitch.org/support/ovscon2020/slides/OVS-CONF-2020-OVN-WITH-DYNAMIC-ROUTING.pdf
> > [2]:
> https://docs.openstack.org/neutron/latest/admin/config-bgp-dynamic-routing.html
> > [3]:
> https://ltomasbo.wordpress.com/2021/02/04/openstack-networking-with-bgp/
> > [4]:
> https://specs.openstack.org/openstack/tripleo-specs/specs/wallaby/triplo-bgp-frrouter.html
> > [5]:
> https://github.com/openstack/tripleo-heat-templates/blob/master/environments/neutron-bgpvpn.yaml
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20211012/e4d6f16c/attachment-0001.htm>