Open Stack

Hello fellow OpenStackers and Kollars (or Koalas) in particular,

today is the day when I finally sit down to write up my thoughts on
Kolla, and I mean *Kolla*, the container image building project for
OpenStack and friends, nothing more (nor less).

***

Some background on Kolla to get everyone on the same page (or close enough):

Kolla builds images for production use. Kolla is upstream for TripleO
containers and Kolla-Ansible deployment.
These are container images, think OCI and Docker in particular since
Kolla actually relies on Dockerfile format to specify build recipes
and Docker runs building (TripleO runs buildah on the same recipes).

Kolla supports three distributions as bases: CentOS (TripleO does
override this for RHEL as well), Debian and Ubuntu.
In the interim periods Kolla supports two distribution releases to
ease/smooth the transition process for operators (like currently
CentOS 7 and 8 in Train, while Ussuri is 8 only).

What is more, each distro has two flavours (or 'types' as they are
called for now): binary and source.
This 'type' applies only to OpenStack software. Binary means that
Kolla uses downstream packages (rpm/deb) while source means to use pip
and install from official tarballs (or repos if it is master) and PyPI
for deps, utilising venv to keep it separate from distro stuff.

Finally, there is particular target architecture: x86_64, aarch64 and ppc64le.

All of above affect the support matrix which is based on 3 or 4
dimensions (distro, flavour, arch and sometimes distro version). [1]

Kolla offers a high level of customisability via various overrides
levels. See docs for details [2]

Kolla engine offers hierarchical approach to image building, under the
assumption that more than one image is often deployed on the same
machine so layer sharing is beneficial.
Kolla helps collect sources for OpenStack projects to build desired
versions of them.

All recipes are templated using Jinja2 syntax.

Images contain both run-time and, mostly in the case of 'source'
flavour, build-time tools and libraries.

*** That's it for the background. :-)

So where is the problem you might ask? Oh, there are plenty.

The general is that Kolla has lots of logic in templates.
Thankfully, Kolla has macros for most stuff but still Jinja2
limitations make it hard to document exceptions (no inline comments in
arrays anyone?).
We lose visibility into real dependency graphs and may be easily
reinstalling same stuff, any optimisations are limited and would
require parsing of both Jinja2 and Dockerfile syntax.
In general the current approach is ugly and becomes unwieldy (e.g.
getting warnings about empty continuation lines).
There is layering hell: dependency on long &&ed commands which are
templated out to avoid useless layers.

Support matrix is great for an overview.
There is a hidden layer to it though, different combinations may
support different features.
This is not documented so far and not so easy to follow from sources.
This stems from the fact that different extra components have
different availability in distros, but this is not easily apparent
from sources.
Also, there is no way to turn feature on/off. You might do an
override, but then again, which to keep, which not to? Which is which?
Oh!

Finally, the images tend to be heavy due to inclusion of build-time deps.

*** That's it for issues (hopefully!)

My idea is to reuse Kolla engine where it shines: sources collection,
system of plugins, hierarchical building; but replace the part that
smells - Jinja2 templating.
Giving up on Jinja2 might encompass giving up on Dockerfile syntax,
but that is optional and depends on what makes it more amenable to
avoid further pitfalls.

With giving up on Jinja2, the idea is to generate building recipes
fully programmatically from Python.
It would be possible to introduce "Features" - sets of packages to
install based on the (distro, arch) tuple.
This would result in more flexibility - turning them on/off (some
could be optional, some not).
There could be more than one optimization strategy regarding when
packages get installed: you want only standalone blah-blah? Then Kolla
won't be installing XYZ and ABC just because ugma-ugma and
tickle-tickle require them and you "could save some space" (TM).
In the same vein, features could declare which components are
build-time and which are run-time and this would make it
straightforward to separate the sides.
The above effort could well be coordinated with different projects to
reuse bindep contents. So far Kolla does not use bindep because it
often installs too much and not enough at the same time.
Do note it would still be bindep-less for external services.

There would still need to exist a general mechanism for providing
custom command executions required by some images.

For contributors and cores this new approach would bring more sanity
as to the scope of proposed changes.
Also, it would be possible to get quick insight into feature support
and autogenerate docs for that as well.
Similarly, current concept of unbuildable images would no longer be
required because unbuildability would be dictated by lack of support
for a required feature.

***

Looking forward to your opinions/thoughts.

***

[1] https://docs.openstack.org/kolla/train/support_matrix.html
[2] https://docs.openstack.org/kolla/train/admin/image-building.html

-yoctozepto