<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p><b>Premise:</b><b><br>
</b><br>
While working with customers, we have realized:<br>
<br>
- They want to use containers but are wary of using the same host
kernel for multiple containers.<br>
- They already have a significant investment (including skills) in
OpenStack's Virtual Machine workflow and would like to re-use it
as much as possible.<br>
- They are very interested in using docker images.<br>
<br>
There are some existing approaches like Hyper, Secure Containers
workflows which already tries to address the first point. But we
wanted to arrive at an approach that addresses all the above three
in context of OpenStack Nova with minimalist changes.<br>
<br>
<b><br>
</b><b>Design Considerations:</b><br>
<br>
We tried a few experiments with the present libvirt driver in nova
to accomplish a work flow to deploy containers inside virtual
machines in OpenStack via Nova.<br>
<br>
The fundamental premise of our approach is to run a single
container encapsulated in a single VM. This VM image just has a
bare minimum operating system required to run it.</p>
<p>The container filesystem comes from the docker image.<br>
<br>
We would like to get the feedback on the below approaches from the
community before proposing this as a spec or blueprint.<br>
<br>
<br>
<b>Approach 1</b><br>
<br>
User workflow:<br>
<br>
1. The docker image is obtained in the form of a tar file.<br>
2. Upload this tar file in glance. This support is already there
in glance were a container-type of docker is supported.<br>
3. Use this image along with nova libvirt driver to deploy a
virtual machine.<br>
<br>
Following are some of the changes to the OpenStack code that
implements this approach:<br>
<br>
1. Define a new conf parameter in nova called – <i>base_vm_image</i>=/var/lib/libvirt/images/baseimage.qcow2<br>
This option is used to specify the base VM image.<br>
<br>
2. define a new <i>sub_virt_type</i> = container in nova conf.
Setting this parameter will ensure mounting of the container
filesystem inside the VM.<br>
Unless qemu and kvm are used as virt_type – this workflow will not
work at this moment.<br>
<br>
3. In the virt/libvirt/driver.py we do the following based on the
sub_virt_type = container:<br>
<br>
- We create a qcow2 disk from the <i>base_vm_image</i> and expose
that 'disk' as the boot disk for the virtual machine.<br>
Note – this is very similar to a regular virtual machine boot
minus the fact that the image is not downloaded from<br>
glance but instead it is present on the host.</p>
<p><br>
</p>
<p>- We download the docker image into the <i>/var/lib/nova/instances/_base
directory</i> and then for each new virtual machine boot – we
create a new directory <i>/var/lib/nova/instances/<instance_uuid></i>
as it's and copy the docker filesystem to it. Note – there are
subsequent improvements to this idea that could be performed
around the lines of using a union filesystem approach.</p>
<p>- The step above allows each virtual machine to have a different
copy of the filesystem.</p>
<p>- We create a '<i>passthrough</i>' mount of the filesystem via
libvirt. This code is also present in the nova libvirt driver and
we just trigger it based on our sub_virt_type parameter.<br>
<br>
4. A cloud init – userdata is provided that looks somewhat like
this:<br>
<i><br>
</i><i>runcmd:</i><i><br>
</i><i> - mount -t 9p -o trans=virtio share_dir /mnt</i><i><br>
</i><i> - chroot /mnt /bin/<command_to_run></i><br>
<br>
The <i>command_to_run </i>is usually the entrypoint to for the
docker image.<br>
<br>
There could be better approaches to determine the entrypoint as
well (say from docker image metadata).<br>
<br>
<b><br>
</b><b>Approach 2.</b><br>
<br>
In this approach, the workflow remains the same as the first one
with the exception that the<br>
docker image is changed into a qcow2 image using a tool like
virt-make-fs before uploading it to glance, instead of a tar file.<br>
<br>
A tool like virt-make-fs can convert a tar file to a qcow2 image
very easily.<br>
<br>
This image is then downloaded on the compute node and a qcow2 disk
is created/attached to the virtual machine that boots using the <i>base_vm_image</i>.<br>
<br>
<br>
<b>Approach 3</b><br>
<br>
A custom qcow2 image is created using kernel, initramfs and the
docker image and uploaded to glance. No changes are needed in
openstack nova. It boots as a regular VM.<br>
<br>
Changes will be needed in image generation tools and will involve
few additional tasks from an operator point of view.</p>
<p><br>
</p>
<p>I look forward to your comments/suggestions on the above.</p>
<p><br>
</p>
<p>Thanks,</p>
<p>Sudipto<br>
</p>
</body>
</html>