Open Stack

On Wed, 29 Oct 2014 12:21:10 AM Fred Baker wrote:
> On Oct 28, 2014, at 4:59 PM, Angus Lees <gus at inodes.org> wrote:
> > On Tue, 28 Oct 2014 09:07:03 PM Rohit Agarwalla wrote:
> >> Agreed. The way I'm thinking about this is that tenants shouldn't care
> >> what
> >> the underlying implementation is - L2 or L3. As long as the connectivity
> >> requirements are met using the model/API, end users should be fine. The
> >> data center network design should be an administrators decision based on
> >> the implementation mechanism that has been configured for OpenStack.
> > 
> > I don't know anything about Project Calico, but I have been involved with
> > running a large cloud network previously that made heavy use of L3
> > overlays.
> > 
> > Just because these points weren't raised earlier in this thread:  In my
> > experience, a move to L3 involves losing:
> > 
> > - broadcast/multicast.  It's possible to do L3 multicast/IGMP/etc, but
> > that's a whole can of worms - so perhaps best to just say up front that
> > this is a non-broadcast network.
> > 
> > - support for other IP protocols.
> > 
> > - various "L2 games" like virtual MAC addresses, etc that NFV/etc people
> > like.
> I’m a little confused. IP supports multicast. It requires a routing
> protocol, and you have to “join” the multicast group, but it’s not out of
> the picture.

Agreed, you absolutely can do multicast and broadcast on an L3 overlay 
network.  I was just saying that IGMP support tends to be a lot more 
inconsistent and flaky across vendors compared with L2 multicast (which pretty 
much always works).

Further, if the goal of moving to routed L3 is to allow a "network" to span 
more geographically diverse underlying networks, then we might want to 
administratively prohibit broadcast due to its increased cost and it no longer 
being a hard requirement for basic functionality (no need for ARP/DHCP 
anymore!).

If we're foregoing an L2 abstraction and moving to L3, I was merely suggesting 
it might also be reasonable to say that broadcast/multicast are not supported 
and thus the requirements on the underlying infrastructure can be drastically 
reduced.  Non-broadcast L3 overlay networks are common and prove to be useful 
for just about every task except mDNS/WINS discovery, which everyone is rather 
happy to leave behind ;)

> What other “IP” protocols do you have in mind? Are you thinking about
> IPX/CLNP/etc? Or are you thinking about new network layers?

eg: If the underlying L3 network only supported IPv4, then it would be 
impossible to run IPv6 (without yet another overlay network).  With a L2 
abstraction, theoretically any IP protocol can be used.

> I’m afraid the L2 games leave me a little cold. We have been there, such as
> with DECNET IV. I’d need to understand what you were trying to achieve
> before I would consider that a loss.

Sure, just listing it as one of the changes for completeness.

"Traditional" network devices often use VRRP or similar for HA failover, and 
so NFV-on-L3 would need to use some alternative (failover via overlapping BGP 
advertisements, for example, is easy and works well, so this isn't hard - just 
different).

> > We gain:
> > 
> > - the ability to have proper hierarchical addressing underneath (which is
> > a
> > big one for scaling a single "network").  This itself is a tradeoff
> > however - an efficient/strict hierarchical addressing scheme means VMs
> > can't choose their own IP addresses, and VM migration is
> > messy/limited/impossible.
> 
> It does require some variation on a host route, and it leads us to ask about
> renumbering. The hard part of VM migration is at the application layer, not
> the network, and is therefore pretty much the same.
> > - hardware support for dynamic L3 routing is generally universal, through
> > a
> > small set of mostly-standard protocols (BGP, ISIS, etc).
> > 
> > - can play various "L3 games" like BGP/anycast, which is super useful for
> > geographically diverse services.
> > 
> > 
> > It's certainly a useful tradeoff for many use cases.  Users lose some
> > generality in return for more powerful cooperation with the provider
> > around
> > particular features, so I sort of think of it like a step halfway up the
> > IaaS-> 
> >> PaaS stack - except for networking.
> > 
> > - Gus
> > 
> >> Thanks
> >> Rohit
> >> 
> >> From: Kevin Benton <blak111 at gmail.com<mailto:blak111 at gmail.com>>
> >> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> >> <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.o
> >> rg
> >> 
> >>>> Date: Tuesday, October 28, 2014 1:01 PM
> >> 
> >> To: "OpenStack Development Mailing List (not for usage questions)"
> >> <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.o
> >> rg
> >> 
> >>>> Subject: Re: [openstack-dev] [neutron][nova] New specs on routed
> >> 
> >> networking
> >> 
> >>> 1. Every packet L3 FIB Lookup : Radix Tree Search, instead of current L2
> >>> Hash/Index Lookup ? 2. Will there be Hierarchical network ?      How
> >>> much
> >>> of the Routes will be imported from external world ? 3. Will there be
> >>> Separate routing domain for overlay network  ? Or it will be mixed with
> >>> external/underlay network ?
> >> 
> >> These are all implementation specific details. Different deployments and
> >> network backends can implement them however they want. What we need to
> >> discuss now is how this model will look to the end-user and API.
> >> 
> >>> 4. What will be the basic use case of this ? Thinking of L3 switching to
> >>> support BGP-MPLS L3 VPN Scenario right from compute node ?
> >> 
> >> I think the simplest use case is just that a provider doesn't want to
> >> deal
> >> with extending L2 domains all over their datacenter.
> >> 
> >> On Tue, Oct 28, 2014 at 12:39 PM, A, Keshava
> >> <keshava.a at hp.com<mailto:keshava.a at hp.com>> wrote: Hi Cory,
> >> 
> >> Yes that is the basic question I have.
> >> 
> >> OpenStack cloud  is ready to move away from Flat L2 network ?
> >> 
> >> 1. Every packet L3 FIB Lookup : Radix Tree Search, instead of current L2
> >> Hash/Index Lookup ? 2. Will there be Hierarchical network ?      How much
> >> of the Routes will be imported from external world ? 3. Will there be
> >> Separate routing domain for overlay network  ? Or it will be mixed with
> >> external/underlay network ? 4. What will be the basic use case of this ?
> >> Thinking of L3 switching to support BGP-MPLS L3 VPN Scenario right from
> >> compute node ?
> >> 
> >> Others can give their opinion also.
> >> 
> >> Thanks & Regards,
> >> keshava
> >> 
> >> -----Original Message-----
> >> From: Cory Benfield
> >> [mailto:Cory.Benfield at metaswitch.com<mailto:Cory.Benfield at metaswitch.com>
> >> ]
> >> Sent: Tuesday, October 28, 2014 10:35 PM
> >> To: OpenStack Development Mailing List (not for usage questions)
> >> Subject: Re: [openstack-dev] [neutron][nova] New specs on routed
> >> networking
> >> 
> >> On Tue, Oct 28, 2014 at 07:44:48, A, Keshava wrote:
> >>> Hi,
> >>> 
> >>> Current Open-stack was built as flat network.
> >>> 
> >>> With the introduction of the L3 lookup (by inserting the routing table
> >>> in forwarding path) and separate 'VIF Route Type' interface:
> >>> 
> >>> At what point of time in the packet processing  decision will be made
> >>> to lookup FIB  during ? For each packet there will additional  FIB
> >>> lookup ?
> >>> 
> >>> How about the  impact on  'inter compute traffic', processed by  DVR  ?
> >>> Here thinking  OpenStack cloud as hierarchical network instead of Flat
> >>> network ?
> >> 
> >> Keshava,
> >> 
> >> It's difficult for me to answer in general terms: the proposed specs are
> >> general enough to allow multiple approaches to building purely-routed
> >> networks in OpenStack, and they may all have slightly different answers
> >> to
> >> some of these questions. I can, however, speak about how Project Calico
> >> intends to apply them.
> >> 
> >> For Project Calico, the FIB lookup is performed for every packet emitted
> >> by
> >> a VM and destined for a VM. Each compute host routes all the traffic
> >> to/from its guests. The DVR approach isn't necessary in this kind of
> >> network because it essentially already implements one: all packets are
> >> always routed, and no network node is ever required in the network.
> >> 
> >> The routed network approach doesn't add any hierarchical nature to an
> >> OpenStack cloud. The difference between the routed approach and the
> >> standard OVS approach is that packet processing happens entirely at layer
> >> 3. Put another way, in Project Calico-based networks a Neutron subnet no
> >> longer maps to a layer 2 broadcast domain.
> >> 
> >> I hope that clarifies: please shout if you'd like more detail.
> >> 
> >> Cory
> >> 
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.or
> >> g>
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >> 
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.or
> >> g>
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >> 
> >> 
> >> 
> >> --
> >> Kevin Benton
> > 
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
 - Gus