Open Stack

I will be there for my lightning talk, and I think armax and kevin_benton
will be there - it would be good to find some time for us to pow-wow, along
with some teleconference so that carl_baldwin and mestery can join in...

Ryan Moats (regXboi)

Mike Dorman <mdorman at godaddy.com> wrote on 08/03/2015 10:07:23 PM:

> From: Mike Dorman <mdorman at godaddy.com>
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>, OpenStack Operators <openstack-
> operators at lists.openstack.org>
> Date: 08/03/2015 10:08 PM
> Subject: Re: [Openstack-operators] [openstack-dev] [Neutron][L3]
> [Large Deployments Team] Representing a networks connected by routers
>
> I hope we can move this idea moving forward.  I was disappointed to see
> the spec abandoned.
>
> Some of us from the large deployers group will be at the Ops Meetup.
Will
> there be any representation from Neutron there that we could discuss with

> more?
>
> Thanks,
> Mike
>
>
>
>
>
> On 8/3/15, 12:27 PM, "Carl Baldwin" <carl at ecbaldwin.net> wrote:
>
> >Kevin, sorry for the delay in response.  Keeping up on this thread was
> >getting difficult while on vacation.
> >
> >tl;dr:  I think it is worth it to talk through the idea of inserting
> >some sort of a "subnet group thing" in the model to which floating ips
> >(and router external gateways) will associate.  It has been on my mind
> >for a long time now.  I didn't pursue it because a few informal
> >attempts to discuss it with others indicated to me that it would be a
> >difficult heavy-lifting job that others may not appreciate or
> >understand.  Scroll to the bottom of this message for a little more on
> >this.
> >
> >Carl
> >
> >On Tue, Jul 28, 2015 at 1:15 AM, Kevin Benton <blak111 at gmail.com> wrote:
> >>>Also, in my proposal, it is more the router that is the grouping
> >>>mechanism.
> >>
> >> I can't reconcile this with all of the points you make in the rest of
> >>your
> >> email. You want the collection of subnets that a network represents,
> >>but you
> >> don't want any other properties of the network.
> >
> >This is closer to what I'm trying to say but isn't quite there.  There
> >are some subnets that should be associated with the segments
> >themselves and there are some that should be associated with the
> >collection of segments.  I want floating IPs that are not tied the an
> >L2 network.  None of the alternate proposals that I'd heard addressed
> >this.
> >
> >>>that the network object is currently the closest thing we have to
> >>> representing the L3 part of the network.
> >>
> >> The L3 part of a network is the subnets. You can't have IP addresses
> >>without
> >> the subnets, you can't have floating IPs without the subnets, etc.
> >
> >You're right but in the current model you can't have IP addresses
> >without the network either which is actually the point I'm trying to
> >make.
> >
> >> A Neutron network is an L2 construct that encapsulates L3 things. By
> >> encapsulating them it also provides an implicit grouping. The routed
> >> networks proposal basically wants that implicit grouping without the
> >> encapsulation or the L2 part.
> >
> >This sounds about right.  I think it is wrong to assume that we need
> >an L2 network to encapsulate L3 things.  I'm feeling restricted by the
> >model and the insistence that a neutron network is a purely L2
> >construct.
> >
> >>>We don't associate floating ips with a network because we want to arp
> >>>for
> >>> them.  You're taking a consequence of the current model and its
> >>>constraints
> >>> and presenting that as the motivation for the model. We do so because

> >>>there
> >>> is no better L3 object to associate it to.
> >>
> >> Don't make assumptions about how people use floating IPs now just
> >>because it
> >> doesn't fit your use-case well. When an external network is
implemented
> >>as a
> >> real Neutron network (leaving external_network_bridge blank like we
> >>suggest
> >> in the networking guide), normal ports can be attached and can
> >> co-exist/communicate with the floating IPs because it behaves as an L2
> >> network exactly as implied by the API. The current model works quite
> >>well if
> >> your fabric can extend the external network everywhere it needs to be.
> >
> >Yes, "when an external network is implemented as a real Neutron
> >network" all of this is true and my proposal doesn't change any of
> >this.  I'm wasn't making any such assumptions.  I acknowledge that the
> >current model works well in this case and didn't intend to change it
> >for current use cases.  It is precisely that because it does not fit
> >my use case well that I'm pursuing this.
> >
> >Notice that a network marked only as external doesn't allow normal
> >tenants to create ports.  It must also be marked shared to allow it.
> >Unless tenants are creating regular ports then they really don't care
> >if arp or anything else L2 is involved because such an external
> >network is meant to give access external to the cloud where L2 is
> >really just an implementation detail.  It is the deployer that cares
> >because of whatever infrastructure (like a gateway router) needs to
> >work with it.  If the L2 is important, then the deployer will not
> >attempt to use an L3 only network, she will use the same kinds of
> >networks as always.
> >
> >The bad assumption here is that floating IPs need an explicit
> >association with an L2 only construct:  tenant's allocate a floating
> >IP by selecting the Neutron network it is recorded in the DB that way.
> >Tenant's aren't even allowed to see the subnets on an external
> >network.  This is counter-intuitive to me because I believe that, in
> >most cases, tenants want a floating IP to get L3 access to the world
> >(or a part of it) that is external to Openstack.  Yet, they can only
> >see the L2 object?  These are the factors that make me view the
> >Neutron network as an L2 + L3 construct.
> >
> >> If you don't want floating IPs to be reachable on the network they are
> >> associated with, then let's stop associating them with a network and
> >>instead
> >> start associating them with a group of subnets from which they
allocate
> >>IPs.
> >
> >Okay.  I'm willing to take a serious look at this.  This isn't merely
> >associating a floating IP with a subnet.  The tenant shouldn't need to
> >know about the individual cidrs of the L3 network and wonder if there
> >is some significant difference or "flavor" of each.  I think we truly
> >need something that represents the group of subnets as you said.
> >
> >>>If we insist on a new object for the L3 part of a network then I'd say

> >>>we
> >>> had better have an L3 only port to connect to it.
> >>
> >> I don't think a new port type is necessary. We can just make the
> >>network ID
> >> nullable for a port to indicate that it isn't attached to a Neutron
> >>network
> >> since it won't be. It would then have a relationship to its associated
> >> subnet via fixed_ips as it does now.
> >
> >Is this really so different from what I'm trying to do with networks?
> >Make the L2 part nullable.
> >
> >>>The subnet is not the L3 object that we're looking for.  You may wish
it
> >>> were but that does not make it so.
> >>
> >> I never said a subnet is what we are looking for. The group of subnets

> >>is
> >> what we seem to be after.
> >
> >Agreed as I stated above.
> >
> >>>Ignoring the forced dependence on L2, the subnets still don't stand
> >>>alone
> >>> to describe just the L3 part, they must be linked to a network to
make
> >>>any
> >>> sense.
> >>
> >> They don't need to be. If we made the network_id nullable on ports and
> >> subnets, we could still have ports associated with subnets. The
network
> >> portion is the L2 portion. You don't want L2 so don't ask for the
> >>network.
> >
> >In your model, should the port be associated with the "group of
> >subnets" at all?  I'm not sure I see a need for it to be directly
> >associated but I haven't thought it all the way through.
> >
> >> I understand that we want a way to reference collections of subnets
and
> >> create ports that allocate IPs from them. Networks provide that now,
but
> >> they imply an L2 domain for the ports, which we don't want. So we are
> >>trying
> >> to change what a network implies for this one special case, which is
> >>going
> >> to have ripple effects everywhere.
> >>
> >> Here are some areas where I can already see we will need
special-casing:
> >>
> >> DHCP agent scheduling - broadcast doesn't work on L3 networks, every
> >>compute
> >> node will need to use the direct tap attachment logic Neil brought up.
> >
> >My proposal only created the ports on the network segments.
> >Admittedly, that is the ugliest part of the proposal but it did
> >obviate the need for DHCP or arp for the port.
> >
> >> DHCP lease generation - a port can't get the normal subnet mask for
the
> >>L3
> >> network it's attached to because it would try to ARP for addresses in
> >>the
> >> same subnet, which are actually somewhere else.
> >
> >See above.
> >
> >> Router interface attachment - what happens when a user attaches one
> >> interface to a regular network and one to an L3 network? Before they
> >>were
> >> all L2 networks so it didn't matter, but now none of the ports will be
> >> reachable on the L3 network without route table manipulation.
> >> (or) Router creation - to avoid the above you can have different
router
> >> types that can only attach to one or the other.
> >
> >Not sure I'm following here.  The ports are all created on the segments.
> >
> >> Every L2 attribute related to networks - we will need logic in the API

> >>that
> >> hides these fields or marks them as invalid and to generate an error
if
> >>the
> >> user tries to update them.
> >> Multi-provider segments - We can't let a user add an L3 segment to a
> >>network
> >> with L2 segments (e.g. VXLAN, VLAN). Same goes for the inverse.
> >> Hierarchical port binding - coordinating ToRs for VXLAN+VLAN is l2
> >>encap. L3
> >> would need route propagation logic instead.
> >
> >All the ports are still connected to L2 network segments so I don't
> >think this is an issue.
> >
> >> Every plugin, service, tool, etc, built on the assumption that a
Neutron
> >> network is L2.
> >
> >ok
> >
> >> Port creation - If you aren't doing the full l3 like Neil's proposal,
> >>you
> >> need to intercept port creation requests and schedule the port to one
> >>of the
> >> underlying regular networks. The port then has a different network_id
> >>than
> >> the one requested, or we have more special-casing to hide that.
> >
> >ok, this was called out and I admit it is the ugliest part of the
> >proposal.
> >
> >> All of those will be branches in the codebase to handle current
Neutron
> >> networks vs L3 networks. If we go down this route, it will be even
worse
> >> than the conditionals we have to support DVR in ML2 because we are
> >>exposing
> >> it via the API. It's technical debt that we will not be able to get
rid
> >>of.
> >
> >I don't think it is nearly as bad as you make it out to be.
> >
> >> I would rather see something to reference a group of subnets that can
be
> >> used for floating IP allocation and port creation in lieu of a network

> >>ID
> >> than the technical debt that conditionally redefining a network will
> >>bring.
> >
> >I'm willing to discuss this further.  In fact, it has been on my mind
> >for a while now.  This is essentially where I started.  I ended up
> >with my current proposal because I perceived a lot more difficulty in
> >doing this than in the proposal I created.  But, your perspective from
> >the other side of the problem is worth considering.
> >
> >I'm glad to see that at least one other person seems to be
> >understanding the problem here.  This will have API and end-user
> >impact is it changes the way that end users interact with floating IPs
> >at least.  It will also affect the way that neutron routers are
> >associated to a network.  Today's use case where a user connects a
> >router to an external network will also change.  To what extent do we
> >support backward compatibility for existing work-flows?  For example,
> >can a user port an existing work-flow to a cloud where the "external
> >network" is now a group of subnets routed among segments instead of a
> >neutron network?
> >
> >Carl
> >
>
>__________________________________________________________________________
> >OpenStack Development Mailing List (not for usage questions)
> >Unsubscribe:
OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150804/55e8f87f/attachment.html>