[Openstack-operators] [openstack-dev] [Neutron][L3] [Large Deployments Team] Representing a networks connected by routers
rmoats at us.ibm.com
Tue Aug 4 13:50:29 UTC 2015
I will be there for my lightning talk, and I think armax and kevin_benton
will be there - it would be good to find some time for us to pow-wow, along
with some teleconference so that carl_baldwin and mestery can join in...
Ryan Moats (regXboi)
Mike Dorman <mdorman at godaddy.com> wrote on 08/03/2015 10:07:23 PM:
> From: Mike Dorman <mdorman at godaddy.com>
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>, OpenStack Operators <openstack-
> operators at lists.openstack.org>
> Date: 08/03/2015 10:08 PM
> Subject: Re: [Openstack-operators] [openstack-dev] [Neutron][L3]
> [Large Deployments Team] Representing a networks connected by routers
> I hope we can move this idea moving forward. I was disappointed to see
> the spec abandoned.
> Some of us from the large deployers group will be at the Ops Meetup.
> there be any representation from Neutron there that we could discuss with
> On 8/3/15, 12:27 PM, "Carl Baldwin" <carl at ecbaldwin.net> wrote:
> >Kevin, sorry for the delay in response. Keeping up on this thread was
> >getting difficult while on vacation.
> >tl;dr: I think it is worth it to talk through the idea of inserting
> >some sort of a "subnet group thing" in the model to which floating ips
> >(and router external gateways) will associate. It has been on my mind
> >for a long time now. I didn't pursue it because a few informal
> >attempts to discuss it with others indicated to me that it would be a
> >difficult heavy-lifting job that others may not appreciate or
> >understand. Scroll to the bottom of this message for a little more on
> >On Tue, Jul 28, 2015 at 1:15 AM, Kevin Benton <blak111 at gmail.com> wrote:
> >>>Also, in my proposal, it is more the router that is the grouping
> >> I can't reconcile this with all of the points you make in the rest of
> >> email. You want the collection of subnets that a network represents,
> >>but you
> >> don't want any other properties of the network.
> >This is closer to what I'm trying to say but isn't quite there. There
> >are some subnets that should be associated with the segments
> >themselves and there are some that should be associated with the
> >collection of segments. I want floating IPs that are not tied the an
> >L2 network. None of the alternate proposals that I'd heard addressed
> >>>that the network object is currently the closest thing we have to
> >>> representing the L3 part of the network.
> >> The L3 part of a network is the subnets. You can't have IP addresses
> >> the subnets, you can't have floating IPs without the subnets, etc.
> >You're right but in the current model you can't have IP addresses
> >without the network either which is actually the point I'm trying to
> >> A Neutron network is an L2 construct that encapsulates L3 things. By
> >> encapsulating them it also provides an implicit grouping. The routed
> >> networks proposal basically wants that implicit grouping without the
> >> encapsulation or the L2 part.
> >This sounds about right. I think it is wrong to assume that we need
> >an L2 network to encapsulate L3 things. I'm feeling restricted by the
> >model and the insistence that a neutron network is a purely L2
> >>>We don't associate floating ips with a network because we want to arp
> >>> them. You're taking a consequence of the current model and its
> >>> and presenting that as the motivation for the model. We do so because
> >>> is no better L3 object to associate it to.
> >> Don't make assumptions about how people use floating IPs now just
> >>because it
> >> doesn't fit your use-case well. When an external network is
> >>as a
> >> real Neutron network (leaving external_network_bridge blank like we
> >> in the networking guide), normal ports can be attached and can
> >> co-exist/communicate with the floating IPs because it behaves as an L2
> >> network exactly as implied by the API. The current model works quite
> >>well if
> >> your fabric can extend the external network everywhere it needs to be.
> >Yes, "when an external network is implemented as a real Neutron
> >network" all of this is true and my proposal doesn't change any of
> >this. I'm wasn't making any such assumptions. I acknowledge that the
> >current model works well in this case and didn't intend to change it
> >for current use cases. It is precisely that because it does not fit
> >my use case well that I'm pursuing this.
> >Notice that a network marked only as external doesn't allow normal
> >tenants to create ports. It must also be marked shared to allow it.
> >Unless tenants are creating regular ports then they really don't care
> >if arp or anything else L2 is involved because such an external
> >network is meant to give access external to the cloud where L2 is
> >really just an implementation detail. It is the deployer that cares
> >because of whatever infrastructure (like a gateway router) needs to
> >work with it. If the L2 is important, then the deployer will not
> >attempt to use an L3 only network, she will use the same kinds of
> >networks as always.
> >The bad assumption here is that floating IPs need an explicit
> >association with an L2 only construct: tenant's allocate a floating
> >IP by selecting the Neutron network it is recorded in the DB that way.
> >Tenant's aren't even allowed to see the subnets on an external
> >network. This is counter-intuitive to me because I believe that, in
> >most cases, tenants want a floating IP to get L3 access to the world
> >(or a part of it) that is external to Openstack. Yet, they can only
> >see the L2 object? These are the factors that make me view the
> >Neutron network as an L2 + L3 construct.
> >> If you don't want floating IPs to be reachable on the network they are
> >> associated with, then let's stop associating them with a network and
> >> start associating them with a group of subnets from which they
> >Okay. I'm willing to take a serious look at this. This isn't merely
> >associating a floating IP with a subnet. The tenant shouldn't need to
> >know about the individual cidrs of the L3 network and wonder if there
> >is some significant difference or "flavor" of each. I think we truly
> >need something that represents the group of subnets as you said.
> >>>If we insist on a new object for the L3 part of a network then I'd say
> >>> had better have an L3 only port to connect to it.
> >> I don't think a new port type is necessary. We can just make the
> >>network ID
> >> nullable for a port to indicate that it isn't attached to a Neutron
> >> since it won't be. It would then have a relationship to its associated
> >> subnet via fixed_ips as it does now.
> >Is this really so different from what I'm trying to do with networks?
> >Make the L2 part nullable.
> >>>The subnet is not the L3 object that we're looking for. You may wish
> >>> were but that does not make it so.
> >> I never said a subnet is what we are looking for. The group of subnets
> >> what we seem to be after.
> >Agreed as I stated above.
> >>>Ignoring the forced dependence on L2, the subnets still don't stand
> >>> to describe just the L3 part, they must be linked to a network to
> >>> sense.
> >> They don't need to be. If we made the network_id nullable on ports and
> >> subnets, we could still have ports associated with subnets. The
> >> portion is the L2 portion. You don't want L2 so don't ask for the
> >In your model, should the port be associated with the "group of
> >subnets" at all? I'm not sure I see a need for it to be directly
> >associated but I haven't thought it all the way through.
> >> I understand that we want a way to reference collections of subnets
> >> create ports that allocate IPs from them. Networks provide that now,
> >> they imply an L2 domain for the ports, which we don't want. So we are
> >> to change what a network implies for this one special case, which is
> >> to have ripple effects everywhere.
> >> Here are some areas where I can already see we will need
> >> DHCP agent scheduling - broadcast doesn't work on L3 networks, every
> >> node will need to use the direct tap attachment logic Neil brought up.
> >My proposal only created the ports on the network segments.
> >Admittedly, that is the ugliest part of the proposal but it did
> >obviate the need for DHCP or arp for the port.
> >> DHCP lease generation - a port can't get the normal subnet mask for
> >> network it's attached to because it would try to ARP for addresses in
> >> same subnet, which are actually somewhere else.
> >See above.
> >> Router interface attachment - what happens when a user attaches one
> >> interface to a regular network and one to an L3 network? Before they
> >> all L2 networks so it didn't matter, but now none of the ports will be
> >> reachable on the L3 network without route table manipulation.
> >> (or) Router creation - to avoid the above you can have different
> >> types that can only attach to one or the other.
> >Not sure I'm following here. The ports are all created on the segments.
> >> Every L2 attribute related to networks - we will need logic in the API
> >> hides these fields or marks them as invalid and to generate an error
> >> user tries to update them.
> >> Multi-provider segments - We can't let a user add an L3 segment to a
> >> with L2 segments (e.g. VXLAN, VLAN). Same goes for the inverse.
> >> Hierarchical port binding - coordinating ToRs for VXLAN+VLAN is l2
> >>encap. L3
> >> would need route propagation logic instead.
> >All the ports are still connected to L2 network segments so I don't
> >think this is an issue.
> >> Every plugin, service, tool, etc, built on the assumption that a
> >> network is L2.
> >> Port creation - If you aren't doing the full l3 like Neil's proposal,
> >> need to intercept port creation requests and schedule the port to one
> >>of the
> >> underlying regular networks. The port then has a different network_id
> >> the one requested, or we have more special-casing to hide that.
> >ok, this was called out and I admit it is the ugliest part of the
> >> All of those will be branches in the codebase to handle current
> >> networks vs L3 networks. If we go down this route, it will be even
> >> than the conditionals we have to support DVR in ML2 because we are
> >> it via the API. It's technical debt that we will not be able to get
> >I don't think it is nearly as bad as you make it out to be.
> >> I would rather see something to reference a group of subnets that can
> >> used for floating IP allocation and port creation in lieu of a network
> >> than the technical debt that conditionally redefining a network will
> >I'm willing to discuss this further. In fact, it has been on my mind
> >for a while now. This is essentially where I started. I ended up
> >with my current proposal because I perceived a lot more difficulty in
> >doing this than in the proposal I created. But, your perspective from
> >the other side of the problem is worth considering.
> >I'm glad to see that at least one other person seems to be
> >understanding the problem here. This will have API and end-user
> >impact is it changes the way that end users interact with floating IPs
> >at least. It will also affect the way that neutron routers are
> >associated to a network. Today's use case where a user connects a
> >router to an external network will also change. To what extent do we
> >support backward compatibility for existing work-flows? For example,
> >can a user port an existing work-flow to a cloud where the "external
> >network" is now a group of subnets routed among segments instead of a
> >neutron network?
> >OpenStack Development Mailing List (not for usage questions)
OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-operators