On 05.05.2020 16:45, Sean Mooney wrote:
Let's for a minute imagine that each of the raised concerns is addressable. And as a thought experiment, let's put here WHAT has to be addressed for Kolla w/o the need of abandoning it for a custom tooling:
On 03.05.2020 21:26, Alex Schultz wrote:
On Sat, May 2, 2020 at 7:45 AM Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2020-05-01 15:18:13 -0500 (-0500), Kevin Carter wrote:
As you may have seen, the TripleO project has been testing the idea of building container images using a simplified toolchain
[...]
Is there an opportunity to collaborate around the proposed plan for publishing basic docker-image-based packages for OpenStack services?
https://review.opendev.org/720107
Obviously you're aiming at solving this for a comprehensive deployment rather than at a packaging level, just wondering if there's a way to avoid having an explosion of different images for the same services if they could ultimately use the same building blocks. (A cynical part of me worries that distro "party lines" will divide folks on what the source of underlying files going into container images should be, but I'm sure our community is better than that, after all we're all in this together.)
I think this assumes we want an all-in-one system to provide containers. And we don't. That I think is the missing piece that folks don't understand about containers and what we actually need.
I believe the issue is that the overall process to go from zero to an application in the container is something like the following:
1) input image (centos/ubi0/ubuntu/clear/whatever)
* support ubi8 base images
2) Packaging method for the application (source/rpm/dpkg/magic)
* abstract away all the packaging methods (at least above the base image) to some (better?) DSL perhaps im not sure a custome dsl is the anser to any of the issues.
3) dependencies provided depending on item #1 & 2 (venv/rpm/dpkg/RDO/ubuntu-cloud/custom)
* abstract away all the dependencies (atm I can only think of go.mod & Go's vendor packages example, sorry) to some extra DSL & CLI tooling may be we have bindeps which is ment to track all binary depenices in a multi disto way. so that is the solution to package depencies
4) layer dependency declaration (base -> nova-base -> nova-api, nova-compute, etc)
* is already fully covered above, I suppose
5) How configurations are provided to the application (at run time or at build)
(what is missing for the run time, almost a perfection yet?) * for the build time, is already fully covered above, i.e. extra DSL & CLI tooling (my biased example: go mod tidy?) kolla does it all outside the container si this is a non issue really. its done at runtime by design wich is the most flexible desing and the the correct one IMO
6) How application is invoked when container is ultimately launched (via docker/podman/k8s/etc)
* have a better DSL to abstract away all the container runtime & orchestration details beneath
On Tue, 2020-05-05 at 15:56 +0200, Bogdan Dobrelya wrote: this has nothing to do with kolla. this is a consern for ooo or kolla ansible but kolla just proivdes image it does not execute them. a dsl is not useful to adress this.
7) Container build method (docker/buildah/other)
* support for buildah (more fancy abstractions and DSL extenstions ofc!)
buildah support the docker file format as far as i am aware so no we dont need a dsl againg i stongly think that is the wrong approch. we just need to ad a config option to the kolla-build binary and add a second module that will invoke buildah instead of docker. so we would just have to modify https://github.com/openstack/kolla/blob/master/kolla/image/build.py we shoudl not need to modify the docker file templates to support this in any way.
if kolla ansible wanted to also support podman it would jsut need to reimplment https://github.com/openstack/kolla-ansible/blob/master/ansible/library/kolla... to provide the same interface but invoke podman. simiarly you caoudl add a config option to select which module to invoke in a task+
The answer to each one of these is dependent on the expectations of the user or application consuming these containers. Additionally this has to be declared for each dependent application as well (rabbitmq/mariadb/etc). Kolla has provided this at a complexity cost because it needs to support any number of combinations for each of
* have better modularity: offload some of the "combinations" to interested 3rd party maintainers (split repos into pluggable modules) and their own CI/CD.
we had talked about having a kolla-infra repo for non openstack service in the past but the effort was deamed not worth it. so non openstack continer like rabbit mariadb and openvswitch could be split out or we could use comunity provided contaienr but im not sure this is needed.
the source vs binary disticion only appplies ot openstack services it does not apply to infra contianers. regardless of the name those are always built using disto binary packages.
these. Today TripleO doesn't use the build method provided by Kolla anymore because we no longer support docker. This means we only use Kolla to generate Dockerfiles as inputs to other processes. It should
NOTE: there is also kolla startup/config APIs on which TripleO will *have to* rely for the next 3-5 years or so. Its compatibility shall not be violated.
be noted that we also only want Dockerfiles for the downstream because they get rebuilt with yet another different process. So for us, we don't want the container and we want a method for generating the contents of the container.
* and again, have better pluggability to abstract away all the downstream vs upstream specifics (btw, I'm not bought on the new custom tooling can solve this problem in a different way but still using better/simpler DSL & tooling)
plugablity wont help. the downstream issue is just that we need to build the contianer using downstream repos which have patches and in some case we want to add or remove depencices based on what is supported in the product.
if we use bindeps correctly. we can achive this witout needing to add plugabilty and the complexity that would invovle.
if we simpley have a list of bindep lables to install per image and then update all bindep files in the complent repos to have lable per backend we could use a single template with defaults labes that when building downstream could simple be overriden to use the labels we support.
for example if nova had say
libvirt,vmware,xen,ceph we could install all of them by default installing the bindeps for ceph libvirt vmware and ceph. downstream we coudl jsut enable libvirt and ceph since we dont support xen or vmware
you woudl do this via the build config file with a list of lables per image.
in the docker file you would just loop over the lables doing "bindep <lable> | <package mager> install ..." to contorl what got installed.
that could be abstracted behind a macro fairly simply by either extending the existing
source config opts with a labels section https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L288-L... or creating a similar one.
we woud need to create a comuntiy goal to have all service adopt and use bindeps to discribe the deps for the different backends they support but that would be a good goal apart for this discussio
IMHO containers are just glorified packaging (yet again and one that lacks ways of expressing dependencies which is really not beneficial for OpenStack). I do not believe you can or should try to unify the entire container declaration and building into a single application. You could rally around a few different sets of tooling that could provide you the pieces for consumption. e.g. A container file templating engine, a building engine, and a way of expressing/consuming configuration+execution information.
I applaud the desire to try and unify all the things, but as we've
So the final call: have pluggable and modular design, adjust DSL and tooling to meet those goals for Kolla. So that one who doesn't chase for unification, just sets up his own module and plugs it into build pipeline. Hint: that "new simpler tooling for TripleO" may be that pluggable module!
i dont think this is the right direction but that said im not going to be working on ooo or kolla in either case to implement my alternitive. modeularity and plugablity is not the aswer in this case in my view. unifying and simplifying build system so that it can be used downstream with no overrides and minimal configuration cannot be achive by plugins and modules.
That I meant is hiding downstream vs upstream differences into configurable parameters that fit into some (probably versioned) schema. And having those YAML files, for example, sitting in a 3rd side repo with custom build image CI jobs set. Wouldn't that help at all?.. Anyway, my intention was only give a few examples and naive suggestions to illustrate the idea. I wasn't aiming to sound right and win all prizes with a 1st hit. But iterate collaboratively to clarify the real problem scope and possible alternatives, at least for the subject spec in TripleO, at most for Kolla roadmap as well.
seen time and time again when it comes to deployment, configuration and use cases. Trying to solve for all the things ends up having a negative effect on the UX because of the complexity required to handle all the cases (look at tripleo for crying out loud). I believe it's time to stop trying to solve all the things with a giant hammer and work on a bunch of smaller nails and let folks construct their own hammer.
Thanks, -Alex
Either way, if they can both make use of the same speculative container building workflow pioneered in Zuul/OpenDev, that seems like a huge win (and I gather the Kolla "krew" are considering redoing their CI jobs along those same lines as well). -- Jeremy Stanley
-- Best regards, Bogdan Dobrelya, Irc #bogdando