<div dir="ltr">><span style="font-size:12.8000001907349px">Also, in my proposal, it is more the router that is the grouping mechanism.</span><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">I can't reconcile this with all of the points you make in the rest of your email. You want the collection of subnets that a network represents, but you don't want any other properties of the network.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">></span><span style="font-size:12.8000001907349px">that the network object is currently the closest thing we have to representing the L3 part of the network.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div>The L3 part of a network is the subnets. You can't have IP addresses without the subnets, you can't have floating IPs without the subnets, etc.</div><div><br></div><div>A Neutron network is an L2 construct that encapsulates L3 things. By encapsulating them it also provides an implicit grouping. The routed networks proposal basically wants that implicit grouping without the encapsulation or the L2 part. </div><div><br></div><div>><span style="font-size:12.8000001907349px">We don't associate floating ips with a network because we want to arp for them. You're taking a consequence of the current model and its constraints and presenting that as the motivation for the model. </span><span style="font-size:12.8000001907349px">We do so because there is no better L3 object to associate it to.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">Don't make assumptions about how people use floating IPs now just because it doesn't fit your use-case well. When an external network is implemented as a real Neutron network (leaving external_network_bridge blank like we suggest in the networking guide), normal ports can be attached and can co-exist/communicate with the floating IPs because it behaves as an L2 network exactly as implied by the API. The current model works quite well <i>if </i>your fabric can extend the external network everywhere it needs to be.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">If you don't want floating IPs to be reachable on the network they are associated with, then let's stop associating them with a network and instead start associating them with a group of subnets from which they allocate IPs.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">></span><span style="font-size:12.8000001907349px">If we insist on a new object for the L3 part of a network then I'd say we had better have an L3 only port to connect to it.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">I don't think a new port type is necessary. We can just make the network ID nullable for a port to indicate that it isn't attached to a Neutron network since it won't be. It would then have a relationship to its associated subnet via fixed_ips as it does now. </span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">></span><span style="font-size:12.8000001907349px">The subnet is not the L3 object that we're looking for. You may wish it were but that does not make it so.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">I never said a subnet is what we are looking for. The group of subnets is what we seem to be after. </span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">></span><span style="font-size:12.8000001907349px">Ignoring the forced dependence on L2, the subnets still don't stand alone to describe just the L3 part, they must be linked to a network to make any sense.</span><br></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">They don't need to be. If we made the network_id nullable on ports and subnets, we could still have ports associated with subnets. The network portion is the L2 portion. You don't want L2 so don't ask for the network. </span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div>I understand that we want a way to reference collections of subnets and create ports that allocate IPs from them. Networks provide that now, but they imply an L2 domain for the ports, which we don't want. So we are trying to change what a network implies for this one special case, which is going to have ripple effects everywhere. </div><div><br></div><div>Here are some areas where I can already see we will need special-casing:</div><div><ul><li>DHCP agent scheduling - broadcast doesn't work on L3 networks, every compute node will need to use the direct tap attachment logic Neil brought up.</li><li>DHCP lease generation - a port can't get the normal subnet mask for the L3 network it's attached to because it would try to ARP for addresses in the same subnet, which are actually somewhere else.</li><li>Router interface attachment - what happens when a user attaches one interface to a regular network and one to an L3 network? Before they were all L2 networks so it didn't matter, but now none of the ports will be reachable on the L3 network without route table manipulation.</li><li>(or) Router creation - to avoid the above you can have different router types that can only attach to one or the other.</li><li>Every L2 attribute related to networks - we will need logic in the API that hides these fields or marks them as invalid and to generate an error if the user tries to update them.</li><li>Multi-provider segments - We can't let a user add an L3 segment to a network with L2 segments (e.g. VXLAN, VLAN). Same goes for the inverse.</li><li>Hierarchical port binding - coordinating ToRs for VXLAN+VLAN is l2 encap. L3 would need route propagation logic instead.<br></li><li>Every plugin, service, tool, etc, built on the assumption that a Neutron network is L2.</li><li>Port creation - If you aren't doing the full l3 like Neil's proposal, you need to intercept port creation requests and schedule the port to one of the underlying regular networks. The port then has a different network_id than the one requested, or we have more special-casing to hide that.</li></ul><div>All of those will be branches in the codebase to handle current Neutron networks vs L3 networks. If we go down this route, it will be even worse than the conditionals we have to support DVR in ML2 because we are exposing it via the API. It's technical debt that we will not be able to get rid of.</div></div><div><br></div><div>I would rather see something to reference a group of subnets that can be used for floating IP allocation and port creation in lieu of a network ID than the technical debt that conditionally redefining a network will bring.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 27, 2015 at 8:55 PM, Carl Baldwin <span dir="ltr"><<a href="mailto:carl@ecbaldwin.net" target="_blank">carl@ecbaldwin.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><p dir="ltr">On Jul 23, 2015 6:04 PM, "Kevin Benton" <<a href="mailto:blak111@gmail.com" target="_blank">blak111@gmail.com</a>> wrote:<br>
><br>
> > IOW, I don't think what I<br>
> proposed in adding L3 stuff to the network that wasn't already here.<br>
><br>
> The point I'm trying to make is that there isn't any L3 stuff on the network itself. There are L3 things that depend on the network because an L2 network carries them. But the routed network proposes something that depends on a network that doesn't actually exist as an L2 network and it doesn't depend on the network for L2 transit. The network makes no sense other than as a grouping mechanism.</p>
</span><p dir="ltr">There is no L3 stuff explicitly on the network but I'm not saying that there is. There doesn't have to be to support my argument. Also, in my proposal, it is more the router that is the grouping mechanism.</p>
<p dir="ltr">I acknowledge that the proposal ignores the L2 parts of the network. That isn't ideal. But, I stand by the point that I'm attempting to make: that the network object is currently the closest thing we have to representing the L3 part of the network. It may not have been the original intention but it is the reality. My proposal isn't creating this situation, it is merely shedding light on it.</p><span class="">
<p dir="ltr">> Even in the floating IP case, you allocate them from a network because the network actually carries them. If you attach a VM to that network it can arp for them and all because it is an L2 network.</p>
</span><p dir="ltr">We don't associate floating ips with a network because we want to arp for them. You're taking a consequence of the current model and its constraints and presenting that as the motivation for the model. We do so because there is no better L3 object to associate it to. The ability to arp for them on an L2 network is inconsequential in a lot of applications. We have it only because that is the only way we support an external network.</p>
<p dir="ltr">A better L3 object might be nice. Floating ips would come from the L3 network. If there is an L2 network underneath, we could do the arp thing if you care about that. The model might be better that way. However, adding it to the model will require heavy lifting with big api impact and significant database migration. We have to look at the relationships of all the basic objects: ports, networks, subnets, and floating ips. Do we need an L3 port, for example? If we insist on a new object for the L3 part of a network then I'd say we had better have an L3 only port to connect to it. This stuff is hard. While it would be nice, does it have to block any progress on L3 network? I'm sure many purists will say yes but I'm not convinced yet.</p>
<p dir="ltr">The subnet is not the L3 object that we're looking for. You may wish it were but that does not make it so. A subnet must have one and only one cidr. A cidr does not an L3 network make. The subnet only makes sense in the context of the network object to which it is associated. That is not because of any of the L2 attributes of the network. Ignoring the forced dependence on L2, the subnets still don't stand alone to describe just the L3 part, they must be linked to a network to make any sense. This is why I continue to argue that the network is effectively our L2 and L3 lumped together. </p>
<p dir="ltr">Imagine there were some new L3 network object in between the L2 and the cidrs. It could associate with the L2 object only if needed. But, as I say, inserting this new object has a big cost. I'm not against doing this modeling work if there is demand for it. It might be fun. But, I know it will be a thankless job that will take an entire cycle at least and will require changes to the api, client, horizon, etc.</p><span class="HOEnZb"><font color="#888888">
<p dir="ltr">Carl</p>
</font></span><br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div>Kevin Benton</div></div>
</div>