[all] [kolla] Kolla Builder - next generation

Radosław Piliszek radoslaw.piliszek at gmail.com
Wed Apr 22 13:11:07 UTC 2020

On Tue, Apr 21, 2020 at 10:59 AM Mark Goddard <mark at stackhpc.com> wrote:
> On Sat, 18 Apr 2020 at 15:10, Radosław Piliszek
> <radoslaw.piliszek at gmail.com> wrote:
> >
> > My idea is to reuse Kolla engine where it shines: sources collection,
> > system of plugins, hierarchical building; but replace the part that
> > smells - Jinja2 templating.
> > Giving up on Jinja2 might encompass giving up on Dockerfile syntax,
> > but that is optional and depends on what makes it more amenable to
> > avoid further pitfalls.
> >
> > With giving up on Jinja2, the idea is to generate building recipes
> > fully programmatically from Python.
> There would be some nice advantages to this, not least it would be
> Python rather than Jinja. Also it would be nicely unit testable.

Agreed, my thoughts exactly.

> However, Jinja does give us some nice properties. Dockerfiles are
> relatively WYSIWIG, and this keeps the barrier to contribution low.
> Often the most that needs to be done to help a contributor get started
> with kolla is to say 'here is the Dockerfile.j2 for the image you want
> to change'.

Fair point, however it comes at the cost of maintaibility and deeper
analysis as we all agreed.

> As Sean rightly pointed out, we have wedded ourselves to Jinja by
> exposing it as a customisation point, through blocks and overrides.
> We'd need to replicate that.

Yes, I reread my mail and noticed I really forgot to mention it.
I already partially answered about that in reply to Sean.
The point is it could actually be easier to replicate than currently.
Why? Because we would have programmatically generated names, instead of
hoping not to make typos in the templates (which happened).

If there was a real problem with some override strategy, at worst we
would have to
provide a translation layer of sorts.
I can't imagine it now though, except for providing completely own
which could just be maintained as legacy mode I guess?

> A potential substitute for unit testing could be to generate every
> supported combination of Dockerfiles, and keep them in the repository.
> These would need to be updated with each change, but would allow us to
> see the final content of the images for each distro. Even better than
> this would be to use TestInfra [2] to validate various expectations
> about image content (e.g. virtualenv uses py3.6, package foo is
> installed).

Oh, that is a nice one indeed. Never heard of TestInfra before.
This might be a good next step to validate that packages install
expected contents.

> > It would be possible to introduce "Features" - sets of packages to
> > install based on the (distro, arch) tuple.
> > This would result in more flexibility - turning them on/off (some
> > could be optional, some not).
> > There could be more than one optimization strategy regarding when
> > packages get installed: you want only standalone blah-blah? Then Kolla
> > won't be installing XYZ and ABC just because ugma-ugma and
> > tickle-tickle require them and you "could save some space" (TM).
> > In the same vein, features could declare which components are
> > build-time and which are run-time and this would make it
> > straightforward to separate the sides.
> This could be neat, although it would be difficult to test and keep
> working, and would only increase our support matrix. You could do it
> with Jinja, although it might get a bit unwieldy.

It would be unit-testable, official images would cater for the general
usage pattern.

> > The above effort could well be coordinated with different projects to
> > reuse bindep contents. So far Kolla does not use bindep because it
> > often installs too much and not enough at the same time.
> > Do note it would still be bindep-less for external services.
> I like the idea of using bindep for source images, in combination with
> multi-stage builds. If we could reduce the length of our package
> dependency lists by relying on bindep, that would be a nice win.
> Ideally, between distro package dependencies and bindep, we would need
> to specify few additional packages.

I think we all agree here.

> >
> > There would still need to exist a general mechanism for providing
> > custom command executions required by some images.
> >
> > For contributors and cores this new approach would bring more sanity
> > as to the scope of proposed changes.
> > Also, it would be possible to get quick insight into feature support
> > and autogenerate docs for that as well.
> > Similarly, current concept of unbuildable images would no longer be
> > required because unbuildability would be dictated by lack of support
> > for a required feature.
> I'm not sure what you mean by 'feature' here, but I'd say the most
> common reason an image is unbuildable currently is that the main
> package required by the image is unavailable, rather than some
> ancillary package required for a particular feature. Modifying
> unbuildable images to include unavailable features could be
> interesting, but would be an extension of current behaviour.

A very particular example for the moment would be cyborg-agent, which
became unbuildable because
OPAE SDK was not available for the supported platforms, while it looks
like it is an optional component
but there was no way to specify that reasoning (except for a comment
but it did not make it it seems).

The point is long list of unbuildable images sprinkled with comments
is less manageable than specific requirements
being unmet by potential targets.

> >
> > ***
> >
> > Looking forward to your opinions/thoughts.
> I think there are some good ideas in here, but I feel that the
> cost/benefit of the core proposal to replace Jinja with Python doesn't
> work for me. Specifically, the cost would be high in terms of being a
> significant rewrite of every image and building additional tooling,
> plus the operator headaches of switching to the new customisation
> model. In terms of benefits, it seems they are mostly for contributors
> rather than users.

It is targeted both at contributors and users as it allows more
flexibility on both sides.
I agree it is going to be a considerable amount of work but so is
anything at this point.
Either leave it be and just hope it doesn't make each cycle more
painful or innovate.

> I'd like us to explore some of the pain points raised here, and also
> see if we can determine any others through the kolla klub.

This is indeed interesting in terms of Kolla Klub.
I really wish to get to know how human users use Kolla.
We mostly talk about Kolla Ansible, for obvious reasons, but Kolla is
equally important for production.

> * Can we reduce the image size? Are we installing unnecessary
> packages? (spoiler, yes) Can we use multi stage builds?

We can, we are, we want.

> * Can we improve testing, to make it clearer what the effects of
> changing a particular image would be?

TestInfra sounds good.

> * What would a 'feature' look like with our current tooling? I'd like
> to see a concrete example.

See my example above. Similarly for collectd plugins targeted at Intel
This would also document the projects as well, best done with bindep
e.g. I need NFS client in a few containers; NFS client is a feature here.

> * What can we learn from the proposed goal [3] to add container images
> for each project? Could kolla be used in a more distributed manner
> more amenable to a CI/CD pattern where each project publishes its own
> images? Could we add a 'python' base distro that is based on the
> python:3-slim image? I realise the authors of the goal won't like
> this, but it does add some missing flexibility to their proposal.

bindep is the common denominator, I guess we should push every effort
for the sake of better bindep.
I wouldn't also mind a switch to "common distro" for the source
flavour (likely as an extra for now).


More information about the openstack-discuss mailing list