Open Stack

It should be noted the community has spent a lot of time discussing this
for the past few months, and I hope we have a plan to move forward in
Mitaka with this soon. We're closing in on the right way forward here I
believe, which is huge. Kudo's to Carl for driving this (and I've copied
him here).

On Tue, Aug 4, 2015 at 2:21 PM, Rochelle Grober <rochelle.grober at huawei.com>
wrote:

> This discussion is important for us to be in on.  At a minimum, we should
> be making sure we have the user stories/use cases and perhaps looking for
> sponsored developers to do the work.  Or maybe we just provide the webex
> for interested parties to attend.  Just doing that would send a signal of
> we're here to help;-)
>
> This is one of the user stories I have been talking about with regards to
> "they are out there written up" and we just need to get them into a
> repository.
>
> --Rocky
>
> From: Kyle Mestery [mailto:mestery at mestery.com]
> Sent: Tuesday, August 04, 2015 7:09 AM
> To: Ryan Moats
> Cc: OpenStack Development Mailing List (not for usage questions);
> OpenStack Operators
> Subject: Re: [openstack-dev] [Openstack-operators] [Neutron][L3] [Large
> Deployments Team] Representing a networks connected by routers
>
> Can you also try to have some sort of remote option? I'd like to attend
> this, and I'd like Carl to try and attend as well. Thanks!
>
> On Tue, Aug 4, 2015 at 8:50 AM, Ryan Moats <rmoats at us.ibm.com<mailto:
> rmoats at us.ibm.com>> wrote:
>
> I will be there for my lightning talk, and I think armax and kevin_benton
> will be there - it would be good to find some time for us to pow-wow, along
> with some teleconference so that carl_baldwin and mestery can join in...
>
> Ryan Moats (regXboi)
>
> Mike Dorman <mdorman at godaddy.com<mailto:mdorman at godaddy.com>> wrote on
> 08/03/2015 10:07:23 PM:
>
> > From: Mike Dorman <mdorman at godaddy.com<mailto:mdorman at godaddy.com>>
> > To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev at lists.openstack.org<mailto:
> openstack-dev at lists.openstack.org>>, OpenStack Operators <openstack-
> > operators at lists.openstack.org<mailto:operators at lists.openstack.org>>
> > Date: 08/03/2015 10:08 PM
> > Subject: Re: [Openstack-operators] [openstack-dev] [Neutron][L3]
> > [Large Deployments Team] Representing a networks connected by routers
>
> >
> > I hope we can move this idea moving forward.  I was disappointed to see
> > the spec abandoned.
> >
> > Some of us from the large deployers group will be at the Ops Meetup.
> Will
> > there be any representation from Neutron there that we could discuss with
> > more?
> >
> > Thanks,
> > Mike
> >
> >
> >
> >
> >
> > On 8/3/15, 12:27 PM, "Carl Baldwin" <carl at ecbaldwin.net<mailto:
> carl at ecbaldwin.net>> wrote:
> >
> > >Kevin, sorry for the delay in response.  Keeping up on this thread was
> > >getting difficult while on vacation.
> > >
> > >tl;dr:  I think it is worth it to talk through the idea of inserting
> > >some sort of a "subnet group thing" in the model to which floating ips
> > >(and router external gateways) will associate.  It has been on my mind
> > >for a long time now.  I didn't pursue it because a few informal
> > >attempts to discuss it with others indicated to me that it would be a
> > >difficult heavy-lifting job that others may not appreciate or
> > >understand.  Scroll to the bottom of this message for a little more on
> > >this.
> > >
> > >Carl
> > >
> > >On Tue, Jul 28, 2015 at 1:15 AM, Kevin Benton <blak111 at gmail.com
> <mailto:blak111 at gmail.com>> wrote:
> > >>>Also, in my proposal, it is more the router that is the grouping
> > >>>mechanism.
> > >>
> > >> I can't reconcile this with all of the points you make in the rest of
> > >>your
> > >> email. You want the collection of subnets that a network represents,
> > >>but you
> > >> don't want any other properties of the network.
> > >
> > >This is closer to what I'm trying to say but isn't quite there.  There
> > >are some subnets that should be associated with the segments
> > >themselves and there are some that should be associated with the
> > >collection of segments.  I want floating IPs that are not tied the an
> > >L2 network.  None of the alternate proposals that I'd heard addressed
> > >this.
> > >
> > >>>that the network object is currently the closest thing we have to
> > >>> representing the L3 part of the network.
> > >>
> > >> The L3 part of a network is the subnets. You can't have IP addresses
> > >>without
> > >> the subnets, you can't have floating IPs without the subnets, etc.
> > >
> > >You're right but in the current model you can't have IP addresses
> > >without the network either which is actually the point I'm trying to
> > >make.
> > >
> > >> A Neutron network is an L2 construct that encapsulates L3 things. By
> > >> encapsulating them it also provides an implicit grouping. The routed
> > >> networks proposal basically wants that implicit grouping without the
> > >> encapsulation or the L2 part.
> > >
> > >This sounds about right.  I think it is wrong to assume that we need
> > >an L2 network to encapsulate L3 things.  I'm feeling restricted by the
> > >model and the insistence that a neutron network is a purely L2
> > >construct.
> > >
> > >>>We don't associate floating ips with a network because we want to arp
> > >>>for
> > >>> them.  You're taking a consequence of the current model and its
> > >>>constraints
> > >>> and presenting that as the motivation for the model. We do so because
> > >>>there
> > >>> is no better L3 object to associate it to.
> > >>
> > >> Don't make assumptions about how people use floating IPs now just
> > >>because it
> > >> doesn't fit your use-case well. When an external network is
> implemented
> > >>as a
> > >> real Neutron network (leaving external_network_bridge blank like we
> > >>suggest
> > >> in the networking guide), normal ports can be attached and can
> > >> co-exist/communicate with the floating IPs because it behaves as an L2
> > >> network exactly as implied by the API. The current model works quite
> > >>well if
> > >> your fabric can extend the external network everywhere it needs to be.
> > >
> > >Yes, "when an external network is implemented as a real Neutron
> > >network" all of this is true and my proposal doesn't change any of
> > >this.  I'm wasn't making any such assumptions.  I acknowledge that the
> > >current model works well in this case and didn't intend to change it
> > >for current use cases.  It is precisely that because it does not fit
> > >my use case well that I'm pursuing this.
> > >
> > >Notice that a network marked only as external doesn't allow normal
> > >tenants to create ports.  It must also be marked shared to allow it.
> > >Unless tenants are creating regular ports then they really don't care
> > >if arp or anything else L2 is involved because such an external
> > >network is meant to give access external to the cloud where L2 is
> > >really just an implementation detail.  It is the deployer that cares
> > >because of whatever infrastructure (like a gateway router) needs to
> > >work with it.  If the L2 is important, then the deployer will not
> > >attempt to use an L3 only network, she will use the same kinds of
> > >networks as always.
> > >
> > >The bad assumption here is that floating IPs need an explicit
> > >association with an L2 only construct:  tenant's allocate a floating
> > >IP by selecting the Neutron network it is recorded in the DB that way.
> > >Tenant's aren't even allowed to see the subnets on an external
> > >network.  This is counter-intuitive to me because I believe that, in
> > >most cases, tenants want a floating IP to get L3 access to the world
> > >(or a part of it) that is external to Openstack.  Yet, they can only
> > >see the L2 object?  These are the factors that make me view the
> > >Neutron network as an L2 + L3 construct.
> > >
> > >> If you don't want floating IPs to be reachable on the network they are
> > >> associated with, then let's stop associating them with a network and
> > >>instead
> > >> start associating them with a group of subnets from which they
> allocate
> > >>IPs.
> > >
> > >Okay.  I'm willing to take a serious look at this.  This isn't merely
> > >associating a floating IP with a subnet.  The tenant shouldn't need to
> > >know about the individual cidrs of the L3 network and wonder if there
> > >is some significant difference or "flavor" of each.  I think we truly
> > >need something that represents the group of subnets as you said.
> > >
> > >>>If we insist on a new object for the L3 part of a network then I'd say
> > >>>we
> > >>> had better have an L3 only port to connect to it.
> > >>
> > >> I don't think a new port type is necessary. We can just make the
> > >>network ID
> > >> nullable for a port to indicate that it isn't attached to a Neutron
> > >>network
> > >> since it won't be. It would then have a relationship to its associated
> > >> subnet via fixed_ips as it does now.
> > >
> > >Is this really so different from what I'm trying to do with networks?
> > >Make the L2 part nullable.
> > >
> > >>>The subnet is not the L3 object that we're looking for.  You may wish
> it
> > >>> were but that does not make it so.
> > >>
> > >> I never said a subnet is what we are looking for. The group of subnets
> > >>is
> > >> what we seem to be after.
> > >
> > >Agreed as I stated above.
> > >
> > >>>Ignoring the forced dependence on L2, the subnets still don't stand
> > >>>alone
> > >>> to describe just the L3 part, they must be linked to a network to
> make
> > >>>any
> > >>> sense.
> > >>
> > >> They don't need to be. If we made the network_id nullable on ports and
> > >> subnets, we could still have ports associated with subnets. The
> network
> > >> portion is the L2 portion. You don't want L2 so don't ask for the
> > >>network.
> > >
> > >In your model, should the port be associated with the "group of
> > >subnets" at all?  I'm not sure I see a need for it to be directly
> > >associated but I haven't thought it all the way through.
> > >
> > >> I understand that we want a way to reference collections of subnets
> and
> > >> create ports that allocate IPs from them. Networks provide that now,
> but
> > >> they imply an L2 domain for the ports, which we don't want. So we are
> > >>trying
> > >> to change what a network implies for this one special case, which is
> > >>going
> > >> to have ripple effects everywhere.
> > >>
> > >> Here are some areas where I can already see we will need
> special-casing:
> > >>
> > >> DHCP agent scheduling - broadcast doesn't work on L3 networks, every
> > >>compute
> > >> node will need to use the direct tap attachment logic Neil brought up.
> > >
> > >My proposal only created the ports on the network segments.
> > >Admittedly, that is the ugliest part of the proposal but it did
> > >obviate the need for DHCP or arp for the port.
> > >
> > >> DHCP lease generation - a port can't get the normal subnet mask for
> the
> > >>L3
> > >> network it's attached to because it would try to ARP for addresses in
> > >>the
> > >> same subnet, which are actually somewhere else.
> > >
> > >See above.
> > >
> > >> Router interface attachment - what happens when a user attaches one
> > >> interface to a regular network and one to an L3 network? Before they
> > >>were
> > >> all L2 networks so it didn't matter, but now none of the ports will be
> > >> reachable on the L3 network without route table manipulation.
> > >> (or) Router creation - to avoid the above you can have different
> router
> > >> types that can only attach to one or the other.
> > >
> > >Not sure I'm following here.  The ports are all created on the segments.
> > >
> > >> Every L2 attribute related to networks - we will need logic in the API
> > >>that
> > >> hides these fields or marks them as invalid and to generate an error
> if
> > >>the
> > >> user tries to update them.
> > >> Multi-provider segments - We can't let a user add an L3 segment to a
> > >>network
> > >> with L2 segments (e.g. VXLAN, VLAN). Same goes for the inverse.
> > >> Hierarchical port binding - coordinating ToRs for VXLAN+VLAN is l2
> > >>encap. L3
> > >> would need route propagation logic instead.
> > >
> > >All the ports are still connected to L2 network segments so I don't
> > >think this is an issue.
> > >
> > >> Every plugin, service, tool, etc, built on the assumption that a
> Neutron
> > >> network is L2.
> > >
> > >ok
> > >
> > >> Port creation - If you aren't doing the full l3 like Neil's proposal,
> > >>you
> > >> need to intercept port creation requests and schedule the port to one
> > >>of the
> > >> underlying regular networks. The port then has a different network_id
> > >>than
> > >> the one requested, or we have more special-casing to hide that.
> > >
> > >ok, this was called out and I admit it is the ugliest part of the
> > >proposal.
> > >
> > >> All of those will be branches in the codebase to handle current
> Neutron
> > >> networks vs L3 networks. If we go down this route, it will be even
> worse
> > >> than the conditionals we have to support DVR in ML2 because we are
> > >>exposing
> > >> it via the API. It's technical debt that we will not be able to get
> rid
> > >>of.
> > >
> > >I don't think it is nearly as bad as you make it out to be.
> > >
> > >> I would rather see something to reference a group of subnets that can
> be
> > >> used for floating IP allocation and port creation in lieu of a network
> > >>ID
> > >> than the technical debt that conditionally redefining a network will
> > >>bring.
> > >
> > >I'm willing to discuss this further.  In fact, it has been on my mind
> > >for a while now.  This is essentially where I started.  I ended up
> > >with my current proposal because I perceived a lot more difficulty in
> > >doing this than in the proposal I created.  But, your perspective from
> > >the other side of the problem is worth considering.
> > >
> > >I'm glad to see that at least one other person seems to be
> > >understanding the problem here.  This will have API and end-user
> > >impact is it changes the way that end users interact with floating IPs
> > >at least.  It will also affect the way that neutron routers are
> > >associated to a network.  Today's use case where a user connects a
> > >router to an external network will also change.  To what extent do we
> > >support backward compatibility for existing work-flows?  For example,
> > >can a user port an existing work-flow to a cloud where the "external
> > >network" is now a group of subnets routed among segments instead of a
> > >neutron network?
> > >
> > >Carl
> > >
> >
> >__________________________________________________________________________
> > >OpenStack Development Mailing List (not for usage questions)
> > >Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<
> http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > _______________________________________________
> > OpenStack-operators mailing list
> > OpenStack-operators at lists.openstack.org<mailto:
> OpenStack-operators at lists.openstack.org>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org<mailto:
> OpenStack-operators at lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
> _______________________________________________
> Product-wg mailing list
> Product-wg at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/product-wg
>
>