Open Stack

Wow, a lot to digest in these threads. If I can summarize my understanding of the two proposals. Let me know whether I get this right. There are a couple problems that need to be solved:

 a. Scheduling based on host reachability to the segments
 b. Floating IP functionality across the segments. I am not sure I am clear on this one but it sounds like you want the routers attached to the segments to advertise routes to the specific floating IPs. Presumably then they would do NAT or the instance would assign both the fixed IP and the floating IP to its interface?

In Proposal 1, (a) is solved by associating segments to the front network via a router - that association is used to provide a single hook into the existing API that limits the scope of segment selection to those associated with the front network. (b) is solved by tying the floating IP ranges to the same front network and managing the reachability with dynamic routing.

In Proposal 2, (a) is solved by tagging each network with some meta-data that the IPAM system uses to make a selection. This implies an IP allocation request that passes something other than a network/port to the IPAM subsystem. This fine from the IPAM point of view but there is no corresponding API for this right now. To solve (b) either the IPAM system has to publish the routes or the higher level management has to ALSO be aware of the mappings (rather than just IPAM).

To throw some fuel on the fire, I would argue also that (a) is not sufficient and address availability needs to be considered as well (as described in [1]). Selecting a host based on reachability alone will fail when addresses are exhausted. Similarly, with (b) I think there needs to be consideration during association of a floating IP to the effect on routing. That is, rather than a huge number of host routes it would be ideal to allocate the floating IPs in blocks that can be associated with the backing networks (though we would want to be able to split these blocks as small as a /32 if necessary - but avoid it/optimize as much as possible).

In fact, I think that these proposals are more or less the same - it's just in #1 the meta-data used to tie the backing networks together is another network. This allows it to fit in neatly with the existing APIs. You would still need to implement something prior to IPAM or within IPAM that would select the appropriate backing network.

As a (gulp) third alternative, we should consider that the front network here is in essence a layer 3 domain, and we have modeled layer 3 domains as address scopes in Liberty. The user is essentially saying "give me an address that is routable in this scope" - they don't care which actual subnet it gets allocated on. This is conceptually more in-line with [2] - modeling L3 domain separately from the existing Neutron concept of a network being a broadcast domain.

Fundamentally, however we associate the segments together, this comes down to a scheduling problem. Nova needs to be able to incorporate data from Neutron in its scheduling decision. Rather than solving this with a single piece of meta-data like network_id as described in proposal 1, it probably makes more sense to build out the general concept of utilizing network data for nova scheduling. We could still model this as in #1, or using address scopes, or some arbitrary data as in #2. But the harder problem to solve is the scheduling, not how we tag these things to inform that scheduling.

The optimization of routing for floating IPs is also a scheduling problem, though one that would require a lot more changes to how FIP are allocated and associated to solve.

John

[1] https://review.openstack.org/#/c/180803/
[2] https://bugs.launchpad.net/neutron/+bug/1458890/comments/7




On Jul 21, 2015, at 10:52 AM, Carl Baldwin <carl at ecbaldwin.net<mailto:carl at ecbaldwin.net>> wrote:


On Jul 20, 2015 4:26 PM, "Ian Wells" <ijw.ubuntu at cack.org.uk<mailto:ijw.ubuntu at cack.org.uk>> wrote:
>
> There are two routed network models:
>
> - I give my VM an address that bears no relation to its location and ensure the routed fabric routes packets there - this is very much the routing protocol method for doing things where I have injected a route into the network and it needs to propagate.  It's also pretty useless because there are too many host routes in any reasonable sized cloud.
>
> - I give my VM an address that is based on its location, which only becomes apparent at binding time.  This means that the semantics of a port changes - a port has no address of any meaning until binding, because its location is related to what it does - and it leaves open questions about what to do when you migrate.
>
> Now, you seem to generally be thinking in terms of the latter model, particularly since the provider network model you're talking about fits there.  But then you say:

Actually, both.  For example, GoDaddy assigns each vm an ip from the location based address blocks and optionally one from the routed location agnostic ones.  I would also like to assign router ports out of the location based blocks which could host floating ips from the other blocks.

> On 20 July 2015 at 10:33, Carl Baldwin <carl at ecbaldwin.net<mailto:carl at ecbaldwin.net>> wrote:
>>
>> When creating a
>> port, the binding information would be sent to the IPAM system and the
>> system would choose an appropriate address block for the allocation.

Implicit in both is a need to provide at least a hint at host binding.  Or, delay address assignment until binding.  I didn't mention it because my email was already long.
This is something and discussed but applies equally to both proposals.

> No, it wouldn't, because creating and binding a port are separate operations.  I can't give the port a location-specific address on creation - not until it's bound, in fact, which happens much later.
>
> On proposal 1: consider the cost of adding a datamodel to Neutron.  It has to be respected by all developers, it frequently has to be deployed by all operators, and every future change has to align with it.  Plus either it has to be generic or optional, and if optional it's a burden to some proportion of Neutron developers and users.  I accept proposal 1 is easy, but it's not universally applicable.  It doesn't work with Neil Jerram's plans, it doesn't work with multiple interfaces per host, and it doesn't work with the IPv6 routed-network model I worked on.

Please be more specific.  I'm not following your argument here.  My proposal doesn't really add much new data model.

We've discussed this with Neil at length.  I haven't been able to reconcile our respective approaches in to one model that works for both of us and still provides value.  The routed segments model needs to somehow handle the L2 details of the underlying network.  Neil's model confines L2 to the port and routes to it.  The two models can't just be squished together unless I'm missing something.

Could you provide some links so that I can brush up on your ipv6 routed network model?  I'd like to consider it but I don't know much about it.

> Given that, I wonder whether proposal 2 could be rephrased.
>
> 1: some network types don't allow unbound ports to have addresses, they just get placeholder addresses for each subnet until they're bound
> 2: 'subnets' on these networks are more special than subnets on other networks.  (More accurately, they dont use subnets.  It's a shame subnets are core Neutron, because they're pretty horrible and yet hard to replace.)
> 3: there's an independent (in an extension?  In another API endpoint?) datamodel that the network points to and that IPAM refers to to find a port an address.  Bonus, people who aren't using funky network types can disable this extension.
> 4: when the port is bound, the IPAM is referred to, and it's told the binding information of the port.
> 5: when binding the port, once IPAM has returned its address, the network controller probably does stuff with that address when it completes the binding (like initialising routing).
> 6: live migration either has to renumber a port or forward old traffic to the new address via route injection.  This is an open question now, so I'm mentioning it rather than solving it.

I left out the migration issue from my email also because it also affects both proposals equally.

> In fact, adding that hook to IPAM at binding plus setting aside a 'not set' IP address might be all you need to do to make it possible.  The IPAM needs data to work out what an address is, but that doesn't have to take the form of existing Neutron constructs.

What about the L2 network for each segment?  I suggested creating provider networks for these.  Do you have a different suggestion?

What about distinguishing the bound address blocks from the mobile address blocks?  For example, the address blocks bound to the segments could be from a private space. A router port may get an address from this private space and be the next hop for public addresses.  Or, GoDaddy's model where vms get an address from the segment network and optionally a floating ip which is routed.

Carl

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org<mailto:OpenStack-dev-request at lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150721/3cd27f4d/attachment.html>