Open Stack

On 05/15/2017 11:44 AM, Ian Wells wrote:
> I'm coming to this cold, so apologies when I put my foot in my mouth.
> But I'm trying to understand what you're actually getting at, here -
> other than helpful simplicity - and I'm not following the detail of
> you're thinking, so take this as a form of enquiry.

Thanks for diving in - no foot-mouth worries here. It's a hard topic.

> On 14 May 2017 at 10:02, Monty Taylor <mordred at inaugust.com
> <mailto:mordred at inaugust.com>> wrote:
>
>     First off, we need to define two terms:
>     "external" - an address that can be used for north-south
>     communication off the cloud
>     "internal" - an address that can be used for east-west communication
>     with and only with other things on the same cloud
>
>
> I'm going through the network detail of this and picking out
> shortcomings, so please understand that before you read on.
>
> I think I see what you're trying to accomplish, but the details don't
> add up for me.  The right answer might be 'you don't use this if you
> want fine detailed network APIs' - and that's fine - but I think the
> issue is not coming up with a model that contradicts the fine detail of
> what you can do with a network today and how you can put it to use.

Yes. The _general_ answer is that this is not supposed to attempt to 
describe all of the networking possibilities - but also exactly what you 
sayd, we don't want to _contradict_ those things.

> 1. What if there are more domains of connectivity, since a cloud can be
> connected to multiple domains?  I see this in its current form as
> intended for public cloud providers as much as anything, in which case
> there is probably only one definition of 'external', for instance, but
> if you want to make it more widely useful you could define off-cloud
> routing domain names, of which 'external' (or, in that context,
> 'internet') is one with a very specific meaning.

Right. So 'external' isn't intended to describe destination, as much as 
'does this go to other things' and 'if so how'.

You're totally right - if there are more than one "external" domain, 
this information will be insufficient. I'd expect each of the networks 
in question to be tagged as "external" - but the choice of where they go 
is a time when a user is going to need more information.

Related to an above reponse though - let's say a user has a cloud that 
connects to three network domains that are not in the cloud in question. 
A, B and C. At no point in that user's life is "I want a VM that has 
external connectivity" going to be a sensible request. They will, out of 
necessity, always need to say "I want a VM that can connect to domain A" 
- which is already very nicely handled.

This is definitely inspired by the Public Clouds, since this is one of 
the top 2 things it's super hard for users to figure out and that we 
currently have to paper over for them in client libraries. (image 
upload, fwiw, is the other) But I hope it degrades well for more complex 
private clouds. That is - the existence of (or lack of) such tags on a 
private cloud should not impact the existing experience for those users 
at all.

> 2. What is 'internal', precisely?  It seems to be in-cloud, though I
> don't entirely understand how NAT comes into that.  What of a route to
> the provider's internal network?  How does it apply when I have multiple
> tenant networks that can't talk to each other, when they're provisioned
> for me and I can't create them, and so on?  Why doesn't it apply to IPv6?

Yes, internal is intended to be in cloud. It's intended to handle the 
"please give me a server that can't talk to things that aren't in my 
personal private network" case. Similar to the external case, if a user 
has more than one project networks, there is no way "I want a VM that 
does not have external connectivity" will be a useful thing to say. 
They'll have to say "I want a VM that is attached to the network I 
created for my database tier" - and again that's currently handled nicely.

In the current proposal IPv6 isn't covered in more depth because the 
question with IPv6 is (in my experience - obviously could be wrong) - 
"can I get me some IPv6".

As for NAT - I included internal NAT for completeness because I heard 
that someone added the ability to neutron to be able to get floating ips 
from one neutron private network and use them to connect to a different 
neutron private network. I have not seen this in action myself, nor do I 
want it - but it seemed having an enum value to cover it wasn't expensive.

As to why we cover NAT at all - that's more to do with workloads running 
on the server. Some workloads like to be able to look at the IP stack on 
the server and see their own network information (kerberos comes to 
mind) while others don't care. And some folks may be designing apps that 
assume they're behind NAT of some sort. On some clouds both are 
available, so a user with a preference to form needs to be able to 
express that.

> 3. Why doesn't your format tell me how to get a port/address of the type
> in question?  Do you feel that everything will be consistent in that
> regard?  To my mind it's more useful - at the least - to tell me the
> *identity* of the network I should be using rather than saying 'such a
> thing is possible in the abstract'.

That's in the follow up suggestion for how the tags could get applied to 
neutron networks - but even without it the metadata is useful.

If your request is "I want a server with external connectivity" and the 
info you have about the cloud says "this cloud has direct external and 
does not have nat" - then you can know that your next step is not 
"create a network, a subnet and a router and connect them". If you 
don't, you have to resort to things like "look for a network that has 
either router:external or one of the provider network flags since those 
are typically external routing networks. try to boot a vm on it. If that 
works, awesome. If it doesn't, try to create a network, subnet and router"

So these are a step, but they're not the whole journey for sure. (also, 
more coming soon about those questions)

> [...]
>
>     "get me a server with only an internal ipv4 and please fail if that
>     isn't possible"
>
>       create_server(
>           'my-server', external_network=False, internal_network=True)
>
>
> A comment on all of these: are you considering this to be an argument
> that is acted upon in the library, or available on the server?

This is example/straw-man library code showing how a library could 
expose an interface that makes use of these things.

> Doing this in the library makes more sense to me.  I prefer the idea of
> documenting in machine-readable form how to use the APIs, because it
> means I can use a cloud without the cloud supporting the API.  For many
> clouds, the description could be a static file, but for more complex
> situations it would be possible to generate it programmatically per tenant.

YES! Exactly. In fact, the cloud profile is currently a static file that 
we just keep in a python library. There is another proposal to suggest a 
way that vendors can provide a file directly themselves.

> Doing it the other way could also lead to cloud-specific code, and
> without some clearer specification it might also lead to cloud-specific
> behaviour.

Yup. 1000% agree (it's like you're reading my mind)

> It's also complexity that simply doesn't need to be in the cloud;
> putting it in the application gives an application with a newer library
> the opportunity to use an older cloud.
>
>     2) As information on networks themselves:
>
>     GET /networks.json
>     {
>       "networks": [
>         {
>           "status": "ACTIVE",
>           "name": "GATEWAY_NET_V6",
>           "id": "54753d2c-0a58-4928-9b32-084c59dd20a6",
>           "network-models": [
>             "ipv4-internal-direct",
>             "ipv6-direct"
>           ]
>         },
>
>
> [...]
>
> I think the problem with this as a concept, if this is what you're
> eventually driving towards, is how you would enumerate this for a network.
>
> IPv6 may be routed to the internet (or other domains) or it may not, but
> if it is it's not currently optional to be locally routed and not
> internet routed on a given network as it is for a v4 address to be fixed
> without a floating component.  (You've skipped this by listing only
> ipv6-direct, I think, as an option, where you have ipv4-fixed).
>
> ipv4 may be routed to the internet if a router is connected, but I can
> connect a router after the fact and I can add a floating IP to a port
> after the fact too.  If you're just thinking in terms of 'when starting
> a VM, at this instant in time' then that might not be quite so much of
> an issue.

Yah. This part is MUCH less well sketched out - for the reasons you're 
pointing at. It actually does start to hit on to more difficult semantic 
questions.

However, the intent here is to answer what sort of things you get from a 
network. So, while you can get a non-routed IPv4 on boot and then later 
connect a router and make a floating ip and attach it - that doesn't 
change the fact that the IPv4 you get from _that_ network is an internal 
IPv4. The network that you'd get the floating IP from to attach to the 
IPv4 you got from this network would have ipv4-external-nat associated 
with it.

(It's sort of an attempt to answer the question you asked earlier- if a 
user now knows that they can get internal IPv4, and external IPv4 via 
NAT, which network should they request those services from.)

>     I'm not suggesting putting info on subnets, since one requests
>     connectivity from a network, not a subnet.
>
>
> Not accurate - I can select a subnet on a network, and it can change who
> I can talk to based on routes.  Neutron routers are attached to subnets,
> not networks.

Right. But servers are booted on networks.

> On a final note, this is really more about 'how do I make a port with
> this sort of connectivity' with the next logical step being that many
> VMs only need one port.

Yes - I agree. I'm phrasing it in terms of servers because that's the 
driving use case (and also for the 80% case ports are completely a 
hidden implementation detail)  - but you are absolutely correct - we're 
actually discussing how to get a port that can be connected _to_ via 
what type of access.

It's also important to note that this is very much centered around 
in-bound connectivity "How do things connect to the port in question?" 
Rather than "how is traffic routed from the port in question?" That's 
largely driven by coming from a direction of "I want to run workload X 
that has the need that other types of things with qualities of Y be able 
to connect to it- how do I do that?"

Thanks for engaging! Excellent questions ... I hope my babbling makes 
the intent more rather than less clear. :)

Monty