Open Stack

On 09/25/2014 12:01 AM, Clint Byrum wrote:
> Excerpts from Mike Spreitzer's message of 2014-09-24 22:01:54 -0700:
>> Clint Byrum <clint at fewbar.com> wrote on 09/25/2014 12:13:53 AM:
>>
>>> Excerpts from Mike Spreitzer's message of 2014-09-24 20:49:20 -0700:
>>>> Steven Dake <sdake at redhat.com> wrote on 09/24/2014 11:02:49 PM:
>>>>> ...
>>>> ...
>>>> Does TripleO require container functionality that is not available
>>>> when using the Docker driver for Nova?
>>>>
>>>> As far as I can tell, the quantitative handling of capacities and
>>>> demands in Kubernetes is much inferior to what Nova does today.
>>>>
>>> Yes, TripleO needs to manage baremetal and containers from a single
>>> host. Nova and Neutron do not offer this as a feature unfortunately.
>> In what sense would Kubernetes "manage baremetal" (at all)?
>> By "from a single host" do you mean that a client on one host
>> can manage remote baremetal and containers?
>>
>> I can see that Kubernetes allows a client on one host to get
>> containers placed remotely --- but so does the Docker driver for Nova.
>>
> I mean that one box would need to host Ironic, Docker, and Nova, for
> the purposes of deploying OpenStack. We call it the "undercloud", or
> sometimes the "Deployment Cloud".
>
> It's not necessarily something that Nova/Neutron cannot do by design,
> but it doesn't work now.
>
>>>>> As far as use cases go, the main use case is to run a specific
>>>>> Docker container on a specific Kubernetes "minion" bare metal host.
>> Clint, in another branch of this email tree you referred to
>> "the VMs that host Kubernetes".  How does that square with
>> Steve's text that seems to imply bare metal minions?
>>
> That was in a more general context, discussing using Kubernetes for
> general deployment. Could have just as easily have said "hosts",
> "machines", or "instances".
>
>> I can see that some people have had much more detailed design
>> discussions than I have yet found.  Perhaps it would be helpful
>> to share an organized presentation of the design thoughts in
>> more detail.
>>
> I personally have not had any detailed discussions about this before it
> was announced. I've just dug into the design and some of the code of
> Kubernetes because it is quite interesting to me.
>
>>>> If TripleO already knows it wants to run a specific Docker image
>>>> on a specific host then TripleO does not need a scheduler.
>>>>
>>> TripleO does not ever specify destination host, because Nova does not
>>> allow that, nor should it. It does want to isolate failure domains so
>>> that all three Galera nodes aren't on the same PDU, but we've not really
>>> gotten to the point where we can do that yet.
>> So I am still not clear on what Steve is trying to say is the main use
>> case.
>> Kubernetes is even farther from balancing among PDUs than Nova is.
>> At least Nova has a framework in which this issue can be posed and solved.
>> I mean a framework that actually can carry the necessary information.
>> The Kubernetes scheduler interface is extremely impoverished in the
>> information it passes and it uses GO structs --- which, like C structs,
>> can not be subclassed.
> I don't think this is totally clear yet. The thing that Steven seems to be
> trying to solve is deploying OpenStack using docker, and Kubernetes may
> very well be a better choice than Nova for this. There are some really
> nice features, and a lot of the benefits we've been citing about image
> based deployments are realized in docker without the pain of a full OS
> image to redeploy all the time.

This is precisely the problem I want to solve.  I looked at Nova+Docker 
as a solution, and it seems to me the runway to get to a successful 
codebase is longer with more risk.  That is why this is an experiment to 
see if a Kubernetes based approach would work.  if at the end of the day 
we throw out Kubernetes as a scheduler once we have the other problems 
solved and reimplement Kubernetes in Nova+Docker, I think that would be 
an acceptable outcome, but not something I want to *start* with but 
*finish* with.

Regards
-steve

> The structs vs. classes argument is completely out of line and has
> nothing to do with where Kubernetes might go in the future. It's like
> saying because cars use internal combustion engines they are limited. It
> is just a facet of how it works today.
>
>> Nova's filter scheduler includes a fatal bug that bites when balancing and
>> you want more than
>> one element per area, see https://bugs.launchpad.net/nova/+bug/1373478.
>> However: (a) you might not need more than one element per area and
>> (b) fixing that bug is a much smaller job than expanding the mind of K8s.
>>
> Perhaps. I am quite a fan of set based design, and Kubernetes is a
> narrowly focused single implementation solution, where Nova is a broadly
> focused abstraction layer for VM's. I think it is worthwhile to push
> a bit into the Kubernetes space and see whether the limitations are
> important or not.
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev