<p dir="ltr">There is no guarantee that two separate external networks will have routing connectivity to each other. It seems like your (b) statement implies that. </p>
<div class="gmail_quote">On Sep 19, 2015 3:12 PM, "Neil Jerram" <<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 17/09/15 19:38, Kevin Benton wrote:<br>
> router:external only affects the behavior of Neutron routers. It<br>
> allows them to attach to it with an external gateway interface which<br>
> implies NAT and floating IPs.<br>
<br>
I presume you're talking about the reference implementation here. OK,<br>
but my concern first is about what it _means_, in particular for<br>
instances attached to such a network. Which your next paragraph addresses:<br>
<br>
><br>
> From an instance's perspective, an external network would be no<br>
> different than any other provider network scenario that uses a<br>
> non-Neutron router. Nothing different happens with the routing of<br>
> traffic.<br>
<br>
Right, thanks. It seems to me that you're saying the same thing here as<br>
my (b) below. In case not, please do say more precisely what isn't<br>
correct about my (b) statement.<br>
<br>
><br>
> >Also I believe that (c) is already true for Neutron external networks<br>
> - i.e. it doesn't make sense to assign a floating IP to an instance<br>
> that is directly on an external network. Is that correct?<br>
><br>
> Well not floating IPs from the same external network, but you could<br>
> conceivably have layers where one external network has an internal<br>
> Neutron router interface that leads to another external network via a<br>
> Neutron router.<br>
<br>
Agreed.<br>
<br>
Many thanks for your input in this thread!<br>
<br>
Regards,<br>
Neil<br>
<br>
><br>
><br>
> On Thu, Sep 17, 2015 at 10:17 AM, Neil Jerram<br>
> <<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a> <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a>>> wrote:<br>
><br>
> Thanks so much for your continuing answers; they are really<br>
> helping me.<br>
><br>
> I see your points now about the special casing, and about the semantic<br>
> expectations and internal wiring of a Neutron network being just the<br>
> same for an external network as for non-external. Hence, the<br>
> model for<br>
> an L3-only external network should be the same as it would be for an<br>
> L3-only tenant network, except for the router:external flag (and might<br>
> be along the lines that you've suggested, of a subnet with a null<br>
> network).<br>
><br>
> It still seems that 'router:external true' might be a good match for<br>
> some of the other 'routed' semantics in [1], though, so I'd like to<br>
> drill down more on exactly what 'router:external true' means.<br>
><br>
> A summary of the semantics at [1] is:<br>
> (a) L3-only connectivity between instances attached to the network.<br>
> (b) Data can be routed between between this network and the<br>
> outside, and<br>
> between multiple networks of this type, without needing Neutron<br>
> routers<br>
> (c) Floating IPs are not supported for instances on this network.<br>
> Instead, wherever an instance needs to be routable from, attach it<br>
> to a<br>
> network with a subnet of IP addresses that are routable from that<br>
> place.<br>
><br>
> [1] <a href="https://review.openstack.org/#/c/198439/" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/198439/</a><br>
> <<a href="https://review.openstack.org/#/c/198439/" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/198439/</a>><br>
><br>
> According to [2], router:external "Indicates whether this network is<br>
> externally accessible." Which I think is an exact match for (b) -<br>
> would<br>
> you agree? (Note: it can't mean that every instance on that network<br>
> actually _is_ contactable from outside, because that depends on IP<br>
> addressing, and router:external is a network property, not<br>
> subnet. But<br>
> it can mean that every instance is _potentially_ contactable, without<br>
> the mediation of a Neutron router.)<br>
><br>
> [2] <a href="http://developer.openstack.org/api-ref-networking-v2-ext.html" rel="noreferrer" target="_blank">http://developer.openstack.org/api-ref-networking-v2-ext.html</a><br>
><br>
> Also I believe that (c) is already true for Neutron external<br>
> networks -<br>
> i.e. it doesn't make sense to assign a floating IP to an instance that<br>
> is directly on an external network. Is that correct?<br>
><br>
> In summary, for the semantics that I'm wanting to model, it sounds<br>
> like<br>
> router:external true already gives me 2 of the 3 main pieces. There's<br>
> still serious work needed for (a), but that's really nice news, if I'm<br>
> seeing things correctly (since discovering that instances can be<br>
> attached to an external network).<br>
><br>
> Regards,<br>
> Neil<br>
><br>
><br>
><br>
><br>
> On 17/09/15 17:29, Kevin Benton wrote:<br>
> ><br>
> > Yes, the L2 semantics apply to the external network as well (at<br>
> least<br>
> > with ML2).<br>
> ><br>
> > One example of the special casing is the external_network_bridge<br>
> > option in the L3 agent. That would cause the agent to plug directly<br>
> > into a bridge so none of the normal L2 agent wiring would occur.<br>
> With<br>
> > the L2 bridge_mappings option there is no reason for this to exist<br>
> > anymore because it ignoring network attributes makes debugging a<br>
> > nightmare.<br>
> ><br>
> > >Yes, that makes sense. Clearly the core semantic there is IP.<br>
> I can<br>
> > imagine reasonable variation on less core details, e.g. L2<br>
> broadcast vs.<br>
> > NBMA. Perhaps it would be acceptable, if use cases need it, for<br>
> such<br>
> > details to be described by flags on the external network object.<br>
> ><br>
> > An external network object is just a regular network object with a<br>
> > router:external flag set to true. Any changes to it would have<br>
> to make<br>
> > sense in the context of all networks. That's why I want to make sure<br>
> > that whatever we come up with makes sense in all contexts and isn't<br>
> > just a bolt on corner case.<br>
> ><br>
> > On Sep 17, 2015 8:21 AM, "Neil Jerram"<br>
> <<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a> <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a>><br>
> > <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a><br>
> <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a>>>> wrote:<br>
> ><br>
> > Thanks, Kevin. Some further queries, then:<br>
> ><br>
> > On 17/09/15 15:49, Kevin Benton wrote:<br>
> > ><br>
> > > It's not true for all plugins, but an external network should<br>
> > provide<br>
> > > the same semantics of a normal network.<br>
> > ><br>
> > Yes, that makes sense. Clearly the core semantic there is<br>
> IP. I can<br>
> > imagine reasonable variation on less core details, e.g. L2<br>
> > broadcast vs.<br>
> > NBMA. Perhaps it would be acceptable, if use cases need it,<br>
> for such<br>
> > details to be described by flags on the external network object.<br>
> ><br>
> > I'm also wondering about what you wrote in the recent thread<br>
> with Carl<br>
> > about representing a network connected by routers. I think<br>
> you were<br>
> > arguing that a L3-only network should not be represented by<br>
> a kind of<br>
> > Neutron network object, because a Neutron network has so many L2<br>
> > properties/semantics that it just doesn't make sense, and better<br>
> > to have<br>
> > a different kind of object for L3-only. Do those L2<br>
> > properties/semantics apply to an external network too?<br>
> ><br>
> > > The only difference is that it allows router gateway<br>
> interfaces<br>
> > to be<br>
> > > attached to it.<br>
> > ><br>
> > Right. From a networking-calico perspective, I think that<br>
> means that<br>
> > the implementation should (eventually) support that, and<br>
> hence allow<br>
> > interconnection between the external network and private Neutron<br>
> > networks.<br>
> ><br>
> > > We want to get rid of as much special casing as possible<br>
> for the<br>
> > > external network.<br>
> > ><br>
> > I don't understand here. What 'special casing' do you mean?<br>
> ><br>
> > Regards,<br>
> > Neil<br>
> ><br>
> > > On Sep 17, 2015 7:02 AM, "Neil Jerram"<br>
> > <<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a><br>
> <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a>><br>
> <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a><br>
> <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a>>><br>
> > > <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a><br>
> <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a>><br>
> > <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a><br>
> <mailto:<a href="mailto:Neil.Jerram@metaswitch.com">Neil.Jerram@metaswitch.com</a>>>>> wrote:<br>
> > ><br>
> > > Thanks to the interesting 'default network model'<br>
> thread, I<br>
> > now know<br>
> > > that Neutron allows booting a VM on an external network.<br>
> > :-) I didn't<br>
> > > realize that before!<br>
> > ><br>
> > > So, I'm now wondering what connectivity semantics are<br>
> > expected (or<br>
> > > even<br>
> > > specified!) for such VMs, and whether they're the same<br>
> as -<br>
> > or very<br>
> > > similar to - the 'routed' networking semantics I've<br>
> > described at [1].<br>
> > ><br>
> > > [1]<br>
> > ><br>
> ><br>
> <a href="https://review.openstack.org/#/c/198439/5/doc/source/devref/routed_networks.rst" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/198439/5/doc/source/devref/routed_networks.rst</a><br>
> > ><br>
> > > Specifically I wonder if VM's attached to an external<br>
> network<br>
> > > expect any<br>
> > > particular L2 characteristics, such as being able to L2<br>
> > broadcast to<br>
> > > each other?<br>
> > ><br>
> > > By way of context - i.e. why am I asking this?... The<br>
> > > networking-calico project [2] provides an<br>
> implementation of the<br>
> > > 'routed'<br>
> > > semantics at [1], but only if one suspends belief in<br>
> some of the<br>
> > > Neutron<br>
> > > semantics associated with non-external networks, such as<br>
> > needing a<br>
> > > virtual router to provide connectivity to the outside<br>
> > world. (Because<br>
> > > networking-calico provides that external connectivity<br>
> > without any<br>
> > > virtual router.) Therefore we believe that we need to<br>
> > propose some<br>
> > > enhancement of the Neutron API and data model, so that<br>
> > Neutron can<br>
> > > describe 'routed' semantics as well as all the traditional<br>
> > ones. But,<br>
> > > if what we are doing is semantically equivalent to<br>
> > 'attaching to an<br>
> > > external network', perhaps no such enhancement is<br>
> needed...<br>
> > ><br>
> > > [2]<br>
> > <a href="https://git.openstack.org/cgit/openstack/networking-calico" rel="noreferrer" target="_blank">https://git.openstack.org/cgit/openstack/networking-calico</a><br>
> > <<a href="https://git.openstack.org/cgit/openstack/networking-calico" rel="noreferrer" target="_blank">https://git.openstack.org/cgit/openstack/networking-calico</a>><br>
> > ><br>
> <<a href="https://git.openstack.org/cgit/openstack/networking-calico" rel="noreferrer" target="_blank">https://git.openstack.org/cgit/openstack/networking-calico</a>><br>
> > ><br>
> > > Many thanks for any input!<br>
> > ><br>
> > > Neil<br>
> > ><br>
> > ><br>
> > ><br>
> ><br>
> __________________________________________________________________________<br>
> > > OpenStack Development Mailing List (not for usage<br>
> questions)<br>
> > > Unsubscribe:<br>
> > ><br>
> ><br>
> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
> ><br>
> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
> > ><br>
> ><br>
> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
> > ><br>
> ><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> > ><br>
> ><br>
> ><br>
> ><br>
> __________________________________________________________________________<br>
> > OpenStack Development Mailing List (not for usage questions)<br>
> > Unsubscribe:<br>
> ><br>
> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
> ><br>
> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
> ><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> ><br>
><br>
><br>
> __________________________________________________________________________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe:<br>
> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
><br>
><br>
><br>
> --<br>
> Kevin Benton<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote></div>