Open Stack

>Also, in my proposal, it is more the router that is the grouping mechanism.

I can't reconcile this with all of the points you make in the rest of your
email. You want the collection of subnets that a network represents, but
you don't want any other properties of the network.

>that the network object is currently the closest thing we have to
representing the L3 part of the network.

The L3 part of a network is the subnets. You can't have IP addresses
without the subnets, you can't have floating IPs without the subnets, etc.

A Neutron network is an L2 construct that encapsulates L3 things. By
encapsulating them it also provides an implicit grouping. The routed
networks proposal basically wants that implicit grouping without the
encapsulation or the L2 part.

>We don't associate floating ips with a network because we want to arp for
them.  You're taking a consequence of the current model and its constraints
and presenting that as the motivation for the model. We do so because there
is no better L3 object to associate it to.

Don't make assumptions about how people use floating IPs now just because
it doesn't fit your use-case well. When an external network is implemented
as a real Neutron network (leaving external_network_bridge blank like we
suggest in the networking guide), normal ports can be attached and can
co-exist/communicate with the floating IPs because it behaves as an L2
network exactly as implied by the API. The current model works quite well *if
*your fabric can extend the external network everywhere it needs to be.

If you don't want floating IPs to be reachable on the network they are
associated with, then let's stop associating them with a network and
instead start associating them with a group of subnets from which they
allocate IPs.

>If we insist on a new object for the L3 part of a network then I'd say we
had better have an L3 only port to connect to it.

I don't think a new port type is necessary. We can just make the network ID
nullable for a port to indicate that it isn't attached to a Neutron network
since it won't be. It would then have a relationship to its associated
subnet via fixed_ips as it does now.

>The subnet is not the L3 object that we're looking for.  You may wish it
were but that does not make it so.

I never said a subnet is what we are looking for. The group of subnets is
what we seem to be after.

>Ignoring the forced dependence on L2, the subnets still don't stand alone
to describe just the L3 part, they must be linked to a network to make any
sense.

They don't need to be. If we made the network_id nullable on ports and
subnets, we could still have ports associated with subnets. The network
portion is the L2 portion. You don't want L2 so don't ask for the network.

I understand that we want a way to reference collections of subnets and
create ports that allocate IPs from them. Networks provide that now, but
they imply an L2 domain for the ports, which we don't want. So we are
trying to change what a network implies for this one special case, which is
going to have ripple effects everywhere.

Here are some areas where I can already see we will need special-casing:

   - DHCP agent scheduling - broadcast doesn't work on L3 networks, every
   compute node will need to use the direct tap attachment logic Neil brought
   up.
   - DHCP lease generation - a port can't get the normal subnet mask for
   the L3 network it's attached to because it would try to ARP for addresses
   in the same subnet, which are actually somewhere else.
   - Router interface attachment - what happens when a user attaches one
   interface to a regular network and one to an L3 network? Before they were
   all L2 networks so it didn't matter, but now none of the ports will be
   reachable on the L3 network without route table manipulation.
   - (or) Router creation - to avoid the above you can have different
   router types that can only attach to one or the other.
   - Every L2 attribute related to networks - we will need logic in the API
   that hides these fields or marks them as invalid and to generate an error
   if the user tries to update them.
   - Multi-provider segments - We can't let a user add an L3 segment to a
   network with L2 segments (e.g. VXLAN, VLAN). Same goes for the inverse.
   - Hierarchical port binding - coordinating ToRs for VXLAN+VLAN is l2
   encap. L3 would need route propagation logic instead.
   - Every plugin, service, tool, etc, built on the assumption that a
   Neutron network is L2.
   - Port creation - If you aren't doing the full l3 like Neil's proposal,
   you need to intercept port creation requests and schedule the port to one
   of the underlying regular networks. The port then has a different
   network_id than the one requested, or we have more special-casing to hide
   that.

All of those will be branches in the codebase to handle current Neutron
networks vs L3 networks. If we go down this route, it will be even worse
than the conditionals we have to support DVR in ML2 because we are exposing
it via the API. It's technical debt that we will not be able to get rid of.

I would rather see something to reference a group of subnets that can be
used for floating IP allocation and port creation in lieu of a network ID
than the technical debt that conditionally redefining a network will bring.

On Mon, Jul 27, 2015 at 8:55 PM, Carl Baldwin <carl at ecbaldwin.net> wrote:

> On Jul 23, 2015 6:04 PM, "Kevin Benton" <blak111 at gmail.com> wrote:
> >
> > > IOW, I don't think what I
> > proposed in adding L3 stuff to the network that wasn't already here.
> >
> > The point I'm trying to make is that there isn't any L3 stuff on the
> network itself. There are L3 things that depend on the network because an
> L2 network carries them. But the routed network proposes something that
> depends on a network that doesn't actually exist as an L2 network and it
> doesn't depend on the network for L2 transit. The network makes no sense
> other than as a grouping mechanism.
>
> There is no L3 stuff explicitly on the network but I'm not saying that
> there is.  There doesn't have to be to support my argument.   Also, in my
> proposal, it is more the router that is the grouping mechanism.
>
> I acknowledge that the proposal ignores the L2 parts of the network.  That
> isn't ideal.  But, I stand by the point that I'm attempting to make:  that
> the network object is currently the closest thing we have to representing
> the L3 part of the network.  It may not have been the original intention
> but it is the reality.  My proposal isn't creating this situation, it is
> merely shedding light on it.
>
> > Even in the floating IP case, you allocate them from a network because
> the network actually carries them. If you attach a VM to that network it
> can arp for them and all because it is an L2 network.
>
> We don't associate floating ips with a network because we want to arp for
> them.  You're taking a consequence of the current model and its constraints
> and presenting that as the motivation for the model.  We do so because
> there is no better L3 object to associate it to.  The ability to arp for
> them on an L2 network is inconsequential in a lot of applications.  We have
> it only because that is the only way we support an external network.
>
> A better L3 object might be nice.  Floating ips would come from the L3
> network.  If there is an L2 network underneath, we could do the arp thing
> if you care about that.  The model might be better that way.  However,
> adding it to the model will require heavy lifting with big api impact and
> significant database migration.  We have to look at the relationships of
> all the basic objects:  ports, networks, subnets, and floating ips.  Do we
> need an L3 port, for example?  If we insist on a new object for the L3 part
> of a network then I'd say we had better have an L3 only port to connect to
> it.  This stuff is hard.  While it would be nice, does it have to block any
> progress on L3 network?  I'm sure many purists will say yes but I'm not
> convinced yet.
>
> The subnet is not the L3 object that we're looking for.  You may wish it
> were but that does not make it so.  A subnet must have one and only one
> cidr.  A cidr does not an L3 network make.  The subnet only makes sense in
> the context of the network object to which it is associated.  That is not
> because of any of the L2 attributes of the network.  Ignoring the forced
> dependence on L2, the subnets still don't stand alone to describe just the
> L3 part, they must be linked to a network to make any sense.  This is why I
> continue to argue that the network is effectively our L2 and L3 lumped
> together.
>
> Imagine there were some new L3 network object in between the L2 and the
> cidrs.  It could associate with the L2 object only if needed.  But, as I
> say, inserting this new object has a big cost.  I'm not against doing this
> modeling work if there is demand for it.  It might be fun.  But, I know it
> will be a thankless job that will take an entire cycle at least and will
> require changes to the api, client, horizon, etc.
>
> Carl
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Kevin Benton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150728/ca9e6db9/attachment.html>