Open Stack

Excerpts from Mike Spreitzer's message of 2014-09-24 22:01:54 -0700:
> Clint Byrum <clint at fewbar.com> wrote on 09/25/2014 12:13:53 AM:
> 
> > Excerpts from Mike Spreitzer's message of 2014-09-24 20:49:20 -0700:
> > > Steven Dake <sdake at redhat.com> wrote on 09/24/2014 11:02:49 PM:
> > > > ...
> > > ...
> > > Does TripleO require container functionality that is not available
> > > when using the Docker driver for Nova?
> > > 
> > > As far as I can tell, the quantitative handling of capacities and
> > > demands in Kubernetes is much inferior to what Nova does today.
> > > 
> > 
> > Yes, TripleO needs to manage baremetal and containers from a single
> > host. Nova and Neutron do not offer this as a feature unfortunately.
> 
> In what sense would Kubernetes "manage baremetal" (at all)?
> By "from a single host" do you mean that a client on one host
> can manage remote baremetal and containers?
> 
> I can see that Kubernetes allows a client on one host to get
> containers placed remotely --- but so does the Docker driver for Nova.
> 

I mean that one box would need to host Ironic, Docker, and Nova, for
the purposes of deploying OpenStack. We call it the "undercloud", or
sometimes the "Deployment Cloud".

It's not necessarily something that Nova/Neutron cannot do by design,
but it doesn't work now.

> > 
> > > > As far as use cases go, the main use case is to run a specific 
> > > > Docker container on a specific Kubernetes "minion" bare metal host.
> 
> Clint, in another branch of this email tree you referred to
> "the VMs that host Kubernetes".  How does that square with
> Steve's text that seems to imply bare metal minions?
> 

That was in a more general context, discussing using Kubernetes for
general deployment. Could have just as easily have said "hosts",
"machines", or "instances".

> I can see that some people have had much more detailed design
> discussions than I have yet found.  Perhaps it would be helpful
> to share an organized presentation of the design thoughts in
> more detail.
> 

I personally have not had any detailed discussions about this before it
was announced. I've just dug into the design and some of the code of
Kubernetes because it is quite interesting to me.

> > > 
> > > If TripleO already knows it wants to run a specific Docker image
> > > on a specific host then TripleO does not need a scheduler.
> > > 
> > 
> > TripleO does not ever specify destination host, because Nova does not
> > allow that, nor should it. It does want to isolate failure domains so
> > that all three Galera nodes aren't on the same PDU, but we've not really
> > gotten to the point where we can do that yet.
> 
> So I am still not clear on what Steve is trying to say is the main use 
> case.
> Kubernetes is even farther from balancing among PDUs than Nova is.
> At least Nova has a framework in which this issue can be posed and solved.
> I mean a framework that actually can carry the necessary information.
> The Kubernetes scheduler interface is extremely impoverished in the
> information it passes and it uses GO structs --- which, like C structs,
> can not be subclassed.

I don't think this is totally clear yet. The thing that Steven seems to be
trying to solve is deploying OpenStack using docker, and Kubernetes may
very well be a better choice than Nova for this. There are some really
nice features, and a lot of the benefits we've been citing about image
based deployments are realized in docker without the pain of a full OS
image to redeploy all the time.

The structs vs. classes argument is completely out of line and has
nothing to do with where Kubernetes might go in the future. It's like
saying because cars use internal combustion engines they are limited. It
is just a facet of how it works today.

> Nova's filter scheduler includes a fatal bug that bites when balancing and 
> you want more than
> one element per area, see https://bugs.launchpad.net/nova/+bug/1373478.
> However: (a) you might not need more than one element per area and
> (b) fixing that bug is a much smaller job than expanding the mind of K8s.
> 

Perhaps. I am quite a fan of set based design, and Kubernetes is a
narrowly focused single implementation solution, where Nova is a broadly
focused abstraction layer for VM's. I think it is worthwhile to push
a bit into the Kubernetes space and see whether the limitations are
important or not.