Open Stack

John,

Good questions. Remarks in-line from the Magnum perspective.

On May 9, 2015, at 2:51 AM, John Garbutt <john at johngarbutt.com> wrote:

> On 1 May 2015 at 16:14, Davanum Srinivas <davanum at gmail.com> wrote:
>> Anyone still interested in this work? :)
>> 
>> * there's a stable/kilo branch now (see
>> http://git.openstack.org/cgit/stackforge/nova-docker/).
>> * CI jobs are running fine against both nova trunk and nova's
>> stable/kilo branch.
>> * there's an updated nova-spec to get code back into nova tree (see
>> https://review.openstack.org/#/c/128753/)
> 
> To proxy the discussion from the etherpad onto the ML, we need to work
> out why this lives in nova, given Magnum is the place to do container
> specific things.

To the extent that users want to control Docker containers through the Nova API (without elaborate extensions), I think a stable in-tree nova-docker driver makes complete sense for that.

> LXC in nova means you can do Magnum containers inside Nova containers,
> and some folks like the idea of that. You can also use containers just
> like VMs, for where that makes sense.

Yes, we do want the ability to form Bays where the Nodes are Nova instances created form the libvirt-lxc virt driver. In fact, ideally we’d like to be able to use any Nova instance, regardless of what virt driver was used to produce it. That logic extends to use of the nova-docker virt driver as well. In that case we would have a Nova instance that’s a docker container inside which we have members of a container cluster like Kubernetes or Swarm that produce containers within the nova container instances. I refer to this arrangement as “nested containers”. See more about this below. 

Practically speaking, we may need to initially focus on a very short list of tested virt-drivers. Over time, I’d like to see that list expand to point where we can claim that Magnum is completely virt driver agnostic.

> Now whats the reason for adding the Docker driver, given Nova is
> considering "container specific" APIs out of scope, and expecting
> Magnum to own that kind of thing.

I do think nova-docker should find it’s way into the Nova tree. This makes containers more accessible in OpenStack, and appropriate for use cases where users want to treat containers like they treat virtual machines. On the subject of extending the Nova API to accommodate special use cases of containers that are beyond the scope of the Nova API, I think we should resist that, and focus those container-specific efforts in Magnum. That way, cloud operators can choose whether to use Nova or Magnum for their container use cases depending on the range of features they desire from the API. This approach should also result in less overlap of efforts.

I will also mention that it’s natural to be allergic to the idea of nested virtualization. We all know that creating multiple levels of hardware virtualization leads to bad performance outcomes. However, "nested containers" do not carry that same drawback, because the actual overhead of a Linux cgroup and Kernel Namespeaces are much lighter than a hardware virtualization. There are cases where having a container-in-container setup gives compelling benefits. That’s why I’ll argue vigorously for both Nova and Magnum to be able to produce container instances both at the machine level, and allow Magnum to produce "nested containers” to produce better workload consolidation density. in a setup with no hypervisors at all.

To sum up, I strongly support merging in nova-docker, with the caveat that it operates within the existing Nova API (with few minor exceptions). For features that require API features that are truly container specific, we should land those in Magnum, and keep the Nova API scoped to operations that are appropriate for “all" instance types.

Adrian

> 
> Thanks,
> John
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev