Hi, At the Denver forum during the kolla feedback session [1], the topic of support for podman [2] and buildah [3] was raised. RHEL/CentOS 8 removes support for Docker [4], although presumably it will still be possible to pull down an RPM from https://download.docker.com/linux/. Currently both kolla and kolla-ansible interact with docker via the docker python module [5], which interacts with the docker daemon API. Given that podman and buildah are not daemons, I would expect the usage model to be a bit different. There is a python API for podman [6] which we might be able to use. I understand that Tripleo uses buildah to build images already (please correct me if I'm wrong). How is this achieved with kolla? Perhaps using 'kolla-build --template-only' to generate Dockerfiles then invoking buildah separately? Are you planning to work on adding buildah support to kolla itself? For running services, Tripleo uses Paunch [7] to abstract away the container engine, and it appears Podman support was added here - building CLI argument strings rather than via a python API [8]. For anyone using kolla/kolla-ansible, please provide feedback on how useful/necessary this would be to you. Thanks, Mark [1] https://etherpad.openstack.org/p/DEN-train-kolla-feedback [2] https://podman.io/ [3] https://buildah.io/ [4] https://access.redhat.com/solutions/3696691 [5] https://github.com/docker/docker-py [6] https://github.com/containers/python-podman [7] https://docs.openstack.org/developer/paunch/readme.html [8] https://github.com/openstack/paunch/blob/ecc2047b2ec5eaf39cce119abe1678ac191...
On Wed, Jun 5, 2019 at 5:21 AM Mark Goddard <mark@stackhpc.com> wrote: [...]
I understand that Tripleo uses buildah to build images already (please correct me if I'm wrong). How is this achieved with kolla? Perhaps using 'kolla-build --template-only' to generate Dockerfiles then invoking buildah separately? Are you planning to work on adding buildah support to kolla itself?
That's what we did indeed, we use Kolla to generate Dockerfiles, then call Buildah from tripleoclient to build containers. We have not planned (yet) to port that workflow to Kolla, which would involve some refacto in the build code (last time I checked). I wrote a blog post about it a while ago: https://my1.fr/blog/openstack-containerization-with-podman-part-5-image-buil... -- Emilien Macchi
On Wed, 5 Jun 2019 at 13:28, Emilien Macchi <emilien@redhat.com> wrote:
On Wed, Jun 5, 2019 at 5:21 AM Mark Goddard <mark@stackhpc.com> wrote: [...]
I understand that Tripleo uses buildah to build images already (please correct me if I'm wrong). How is this achieved with kolla? Perhaps using 'kolla-build --template-only' to generate Dockerfiles then invoking buildah separately? Are you planning to work on adding buildah support to kolla itself?
That's what we did indeed, we use Kolla to generate Dockerfiles, then call Buildah from tripleoclient to build containers. We have not planned (yet) to port that workflow to Kolla, which would involve some refacto in the build code (last time I checked).
I wrote a blog post about it a while ago: https://my1.fr/blog/openstack-containerization-with-podman-part-5-image-buil...
Thanks for following up. It wouldn't be a trivial change to add buildah support in kolla, but it would have saved reimplementing the task parallelisation in Tripleo and would benefit others too. Never mind.
-- Emilien Macchi
On Wed, 2019-06-05 at 13:47 +0100, Mark Goddard wrote:
On Wed, 5 Jun 2019 at 13:28, Emilien Macchi <emilien@redhat.com> wrote:
On Wed, Jun 5, 2019 at 5:21 AM Mark Goddard <mark@stackhpc.com> wrote: [...]
I understand that Tripleo uses buildah to build images already (please correct me if I'm wrong). How is this achieved with kolla? Perhaps using 'kolla-build --template-only' to generate Dockerfiles then invoking buildah separately? Are you planning to work on adding buildah support to kolla itself?
That's what we did indeed, we use Kolla to generate Dockerfiles, then call Buildah from tripleoclient to build containers. We have not planned (yet) to port that workflow to Kolla, which would involve some refacto in the build code (last time I checked).
I wrote a blog post about it a while ago: https://my1.fr/blog/openstack-containerization-with-podman-part-5-image-buil...
Thanks for following up. It wouldn't be a trivial change to add buildah support in kolla, but it would have saved reimplementing the task parallelisation in Tripleo and would benefit others too. Never mind. actully im not sure about that buildah should actully be pretty simple to add support for. its been a while but we looksed at swaping out the building with a python script a few years ago https://review.opendev.org/#/c/503882/ and it really did not take that much to enable so simply invoking buildah in a simlar maner should be trivail.
podman support will be harder but again we confied all interaction with docker in kolla-ansibel to be via https://github.com/openstack/kolla-ansible/blob/master/ansible/library/kolla... so we sould jsut need to write a similar module that would work with podman and then select the correct one to use. the interface is a little large but it shoudld reactively mechanical to implement podman supprot.
-- Emilien Macchi
On Wed, Jun 5, 2019 at 8:46 AM Sean Mooney <smooney@redhat.com> wrote:
On Wed, 5 Jun 2019 at 13:28, Emilien Macchi <emilien@redhat.com> wrote:
On Wed, Jun 5, 2019 at 5:21 AM Mark Goddard <mark@stackhpc.com> wrote: [...]
I understand that Tripleo uses buildah to build images already
(please
correct me if I'm wrong). How is this achieved with kolla? Perhaps using 'kolla-build --template-only' to generate Dockerfiles then invoking buildah separately? Are you planning to work on adding buildah support to kolla itself?
That's what we did indeed, we use Kolla to generate Dockerfiles, then call Buildah from tripleoclient to build containers. We have not planned (yet) to port that workflow to Kolla, which would involve some refacto in the build code (last time I checked).
I wrote a blog post about it a while ago:
https://my1.fr/blog/openstack-containerization-with-podman-part-5-image-buil...
Thanks for following up. It wouldn't be a trivial change to add buildah support in kolla, but it would have saved reimplementing the task parallelisation in Tripleo and would benefit others too. Never mind. actully im not sure about that buildah should actully be pretty simple to add support for. its been a while but we looksed at swaping out the building with a python
On Wed, 2019-06-05 at 13:47 +0100, Mark Goddard wrote: script a few years ago https://review.opendev.org/#/c/503882/ and it really did not take that much to enable so simply invoking buildah in a simlar maner should be trivail.
The issue was trying to build the appropriate parallelization logic based on the kolla container build order[0]. We're using the --list-dependencies to get the ordering for the build[1] and then run it through our builder[2]. You wouldn't want to do it serially because it's dramatically slower. Our buildah builder is only slightly slower than the docker one at this point.
podman support will be harder but again we confied all interaction with docker in kolla-ansibel to be via
https://github.com/openstack/kolla-ansible/blob/master/ansible/library/kolla... so we sould jsut need to write a similar module that would work with podman and then select the correct one to use. the interface is a little large but it shoudld reactively mechanical to implement podman supprot.
The podman support is a bit more complex because there is no daemon associated with it. We wrote systemd/podman support into paunch[3] for us to handle the management of the life cycles of the containers. We'd like to investigate switching our invocation of paunch from cli to an ansible plugin/module which might be beneficial for kolla-ansible as well. Thanks, -Alex [0] https://opendev.org/openstack/tripleo-common/src/branch/master/tripleo_commo... [1] https://opendev.org/openstack/tripleo-common/src/branch/master/tripleo_commo... [2] https://opendev.org/openstack/python-tripleoclient/src/branch/master/tripleo... [3] https://opendev.org/openstack/paunch
-- Emilien Macchi
Whats the plan for Kubernetes integration at this point? I keep seeing more and more talk/work on integrating podman and paunch and such but its a lot of work that doesn't apply when switching to Kubernetes? Thanks, Kevin ________________________________ From: Alex Schultz [aschultz@redhat.com] Sent: Wednesday, June 05, 2019 7:59 AM To: Sean Mooney Cc: Mark Goddard; Emilien Macchi; openstack-discuss Subject: Re: [kolla][tripleo] Podman/buildah On Wed, Jun 5, 2019 at 8:46 AM Sean Mooney <smooney@redhat.com<mailto:smooney@redhat.com>> wrote: On Wed, 2019-06-05 at 13:47 +0100, Mark Goddard wrote:
On Wed, 5 Jun 2019 at 13:28, Emilien Macchi <emilien@redhat.com<mailto:emilien@redhat.com>> wrote:
On Wed, Jun 5, 2019 at 5:21 AM Mark Goddard <mark@stackhpc.com<mailto:mark@stackhpc.com>> wrote: [...]
I understand that Tripleo uses buildah to build images already (please correct me if I'm wrong). How is this achieved with kolla? Perhaps using 'kolla-build --template-only' to generate Dockerfiles then invoking buildah separately? Are you planning to work on adding buildah support to kolla itself?
That's what we did indeed, we use Kolla to generate Dockerfiles, then call Buildah from tripleoclient to build containers. We have not planned (yet) to port that workflow to Kolla, which would involve some refacto in the build code (last time I checked).
I wrote a blog post about it a while ago: https://my1.fr/blog/openstack-containerization-with-podman-part-5-image-buil...
Thanks for following up. It wouldn't be a trivial change to add buildah support in kolla, but it would have saved reimplementing the task parallelisation in Tripleo and would benefit others too. Never mind. actully im not sure about that buildah should actully be pretty simple to add support for. its been a while but we looksed at swaping out the building with a python script a few years ago https://review.opendev.org/#/c/503882/ and it really did not take that much to enable so simply invoking buildah in a simlar maner should be trivail.
The issue was trying to build the appropriate parallelization logic based on the kolla container build order[0]. We're using the --list-dependencies to get the ordering for the build[1] and then run it through our builder[2]. You wouldn't want to do it serially because it's dramatically slower. Our buildah builder is only slightly slower than the docker one at this point. podman support will be harder but again we confied all interaction with docker in kolla-ansibel to be via https://github.com/openstack/kolla-ansible/blob/master/ansible/library/kolla... so we sould jsut need to write a similar module that would work with podman and then select the correct one to use. the interface is a little large but it shoudld reactively mechanical to implement podman supprot. The podman support is a bit more complex because there is no daemon associated with it. We wrote systemd/podman support into paunch[3] for us to handle the management of the life cycles of the containers. We'd like to investigate switching our invocation of paunch from cli to an ansible plugin/module which might be beneficial for kolla-ansible as well. Thanks, -Alex [0] https://opendev.org/openstack/tripleo-common/src/branch/master/tripleo_commo... [1] https://opendev.org/openstack/tripleo-common/src/branch/master/tripleo_commo... [2] https://opendev.org/openstack/python-tripleoclient/src/branch/master/tripleo... [3] https://opendev.org/openstack/paunch
-- Emilien Macchi
On Wed, Jun 5, 2019 at 8:47 AM Mark Goddard <mark@stackhpc.com> wrote:
Thanks for following up. It wouldn't be a trivial change to add buildah support in kolla, but it would have saved reimplementing the task parallelisation in Tripleo and would benefit others too. Never mind.
To be fair, at the time I wrote the code in python-tripleoclient the container tooling wasn't really stable and we weren't sure about the directions we would take yet; which is the main reason which drove us to not invest too much time into refactoring Kolla to support a tool that we weren't sure we would end up using in production for the container image building. It has been a few months now and so far it works ok for our needs; so if there is interest in supporting Buildah in Kolla then we might want to do the refactor and of course TripleO would use this new feature. -- Emilien Macchi
On Wed, 5 Jun 2019 at 15:48, Emilien Macchi <emilien@redhat.com> wrote:
On Wed, Jun 5, 2019 at 8:47 AM Mark Goddard <mark@stackhpc.com> wrote:
Thanks for following up. It wouldn't be a trivial change to add buildah support in kolla, but it would have saved reimplementing the task parallelisation in Tripleo and would benefit others too. Never mind.
To be fair, at the time I wrote the code in python-tripleoclient the container tooling wasn't really stable and we weren't sure about the directions we would take yet; which is the main reason which drove us to not invest too much time into refactoring Kolla to support a tool that we weren't sure we would end up using in production for the container image building.
That's fair, sorry to grumble :)
It has been a few months now and so far it works ok for our needs; so if there is interest in supporting Buildah in Kolla then we might want to do the refactor and of course TripleO would use this new feature. -- Emilien Macchi
On Wed, 5 Jun 2019 at 17:31, Mark Goddard <mark@stackhpc.com> wrote:
On Wed, 5 Jun 2019 at 15:48, Emilien Macchi <emilien@redhat.com> wrote:
On Wed, Jun 5, 2019 at 8:47 AM Mark Goddard <mark@stackhpc.com> wrote:
Thanks for following up. It wouldn't be a trivial change to add buildah support in kolla, but it would have saved reimplementing the task parallelisation in Tripleo and would benefit others too. Never mind.
To be fair, at the time I wrote the code in python-tripleoclient the container tooling wasn't really stable and we weren't sure about the directions we would take yet; which is the main reason which drove us to not invest too much time into refactoring Kolla to support a tool that we weren't sure we would end up using in production for the container image building.
That's fair, sorry to grumble :)
It has been a few months now and so far it works ok for our needs; so if there is interest in supporting Buildah in Kolla then we might want to do the refactor and of course TripleO would use this new feature.
If there are resources to do it, I'm sure the Kolla team would be receptive.
-- Emilien Macchi
participants (5)
-
Alex Schultz
-
Emilien Macchi
-
Fox, Kevin M
-
Mark Goddard
-
Sean Mooney