Greetings, Everyone you are going to find your jobs going red atm. My apologies some containers were not pushed manually properly and I'm having to repull and and repush to docker.io atm. We should have this fixed in about 2 hours. Sorry again, and hopefully you'll be back up and running shortly. Thanks
On Mon, Mar 9, 2020 at 8:57 PM Wesley Hayutin <whayutin@redhat.com> wrote:
Greetings,
Everyone you are going to find your jobs going red atm. My apologies some containers were not pushed manually properly and I'm having to repull and and repush to docker.io atm. We should have this fixed in about 2 hours.
Sorry again, and hopefully you'll be back up and running shortly.
Thanks
Quick update.. We're about 1/2 done w/ pushing the new containers for centos-8, the tag is af182654cc32d30ea7f4774eb06ed9fd <https://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo.md5> Hopefully all the mirrors will get seeded quickly. Sorry for the inconvenience.
Greetings, Follow up.. new containers have been pushed to docker.io. Thank you for your patience! On Mon, Mar 9, 2020 at 10:24 PM Wesley Hayutin <whayutin@redhat.com> wrote:
On Mon, Mar 9, 2020 at 8:57 PM Wesley Hayutin <whayutin@redhat.com> wrote:
Greetings,
Everyone you are going to find your jobs going red atm. My apologies some containers were not pushed manually properly and I'm having to repull and and repush to docker.io atm. We should have this fixed in about 2 hours.
Sorry again, and hopefully you'll be back up and running shortly.
Thanks
Quick update.. We're about 1/2 done w/ pushing the new containers for centos-8, the tag is af182654cc32d30ea7f4774eb06ed9fd <https://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo.md5>
Hopefully all the mirrors will get seeded quickly. Sorry for the inconvenience.
On Mar 10, 2020, at 9:24 AM, Wesley Hayutin <whayutin@redhat.com> wrote:
Greetings,
Follow up.. new containers have been pushed to docker.io. Thank you for your patience!
Yay! When you have brainspace after firefighting (always fun) - maybe we should find a time to talk about whether our image building and publishing automation could help you out here. No rush - this is one of those “we’ve got some tools we might be able to leverage to help” - just ping me whenever.
On Mon, Mar 9, 2020 at 10:24 PM Wesley Hayutin <whayutin@redhat.com> wrote:
On Mon, Mar 9, 2020 at 8:57 PM Wesley Hayutin <whayutin@redhat.com> wrote: Greetings,
Everyone you are going to find your jobs going red atm. My apologies some containers were not pushed manually properly and I'm having to repull and and repush to docker.io atm. We should have this fixed in about 2 hours.
Sorry again, and hopefully you'll be back up and running shortly.
Thanks
Quick update.. We're about 1/2 done w/ pushing the new containers for centos-8, the tag is af182654cc32d30ea7f4774eb06ed9fd
Hopefully all the mirrors will get seeded quickly. Sorry for the inconvenience.
On Tue, Mar 10, 2020 at 8:34 AM Monty Taylor <mordred@inaugust.com> wrote:
On Mar 10, 2020, at 9:24 AM, Wesley Hayutin <whayutin@redhat.com> wrote:
Greetings,
Follow up.. new containers have been pushed to docker.io. Thank you for your patience!
Yay!
When you have brainspace after firefighting (always fun) - maybe we should find a time to talk about whether our image building and publishing automation could help you out here. No rush - this is one of those “we’ve got some tools we might be able to leverage to help” - just ping me whenever.
Definitely interested!! Some additional context on what we're up to is helpful as well. Our current tooling is here [1] and we're under the gun w/ getting centos-8 ready for ussuri and DLRN's new component feature [2]. If there are upstream tooling and processes we can incorporate we'd be interested in picking our heads up and listening! Thanks as usual Monty! [1] https://github.com/rdo-infra/ci-config/tree/master/ci-scripts/dlrnapi_promot... [2] https://review.rdoproject.org/r/#/c/24818/ https://trunk.rdoproject.org/centos8-master/component/ https://dlrn.readthedocs.io/en/latest/api.html
On Mon, Mar 9, 2020 at 10:24 PM Wesley Hayutin <whayutin@redhat.com> wrote:
On Mon, Mar 9, 2020 at 8:57 PM Wesley Hayutin <whayutin@redhat.com> wrote: Greetings,
Everyone you are going to find your jobs going red atm. My apologies some containers were not pushed manually properly and I'm having to repull and and repush to docker.io atm. We should have this fixed in about 2 hours.
Sorry again, and hopefully you'll be back up and running shortly.
Thanks
Quick update.. We're about 1/2 done w/ pushing the new containers for centos-8, the tag is af182654cc32d30ea7f4774eb06ed9fd
Hopefully all the mirrors will get seeded quickly. Sorry for the inconvenience.
On Tue, Mar 10, 2020 at 10:41 AM Monty Taylor <mordred@inaugust.com> wrote:
Yay!
When you have brainspace after firefighting (always fun) - maybe we should find a time to talk about whether our image building and publishing automation could help you out here. No rush - this is one of those “we’ve got some tools we might be able to leverage to help” - just ping me whenever.
Hey Monty, The CI team is presently busy with CentOS 8 fires but I would be happy to help and work together on convergence. Maybe I can start by explaining how our process works, then you can do the same and we see where we can collaborate. The TL;DR is that we have built TripleO CLI and Ansible roles to consume Kolla tooling and build our images. 1) How a TripleO user would build an image? By using the "openstack overcloud container image build" command ("overcloud" is here by legacy, please ignore it). The magic happens here: https://opendev.org/openstack/python-tripleoclient/src/branch/master/tripleo... It's basically wrapping out the kolla-build CLI; with proper options for us. In fact, since podman/buildah, we only use kolla-build to render Kolla Dockerfiles templates to merge them with our TripleO overrides: https://opendev.org/openstack/tripleo-common/src/branch/master/container-ima... kolla-build would generate directories for each image and inside you would have their Dockerfiles. We don't use kolla-build to build the containers because Kolla doesn't support Buildah, and none of us has taken the time to do it yet. To build the images from Dockerfiles, we use that code: https://opendev.org/openstack/tripleo-common/src/branch/master/tripleo_commo... It's basically running "buildah bud" with concurrency (to make it faster). This code could be converted to an Ansible module eventually; which could be consumed by more than us. Once images are built, the code runs "buildah push"; to push it to a remote (or local) registry. That's it, that's all. If we resume, we use kolla-build to generate Dockerfiles for our containers (since TripleO images use Kolla format) and then we have our own crap to use Buildah to build & push the image. I guess the second part is something we could share. 2) How TripleO CI builds containers? We have an Ansible role for that: https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-conta... It basically: - Install repositories needed to deploy TripleO - Deploy a local docker registry with ansible-role-container-registry (also used in production when Docker is deployed, so before Stein) - Install and configure Kolla - Runs "openstack overcloud container image build" (which was described earlier) to build, tag and push images I skipped a few details but this is the big picture. I'm sure there is a lot where we can share and I would be more than happy to contribute in that effort, please let me know how it works on your side and we'll find ways to collaborate. Thanks, -- Emilien Macchi
On Tue, Mar 10, 2020 at 19:38 Emilien Macchi wrote:
On Tue, Mar 10, 2020 at 10:41 AM Monty Taylor <mordred@inaugust.com> wrote:
Yay!
When you have brainspace after firefighting (always fun) - maybe we should find a time to talk about whether our image building and publishing automation could help you out here. No rush - this is one of those “we’ve got some tools we might be able to leverage to help” - just ping me whenever.
Hey Monty,
The CI team is presently busy with CentOS 8 fires but I would be happy to help and work together on convergence. Maybe I can start by explaining how our process works, then you can do the same and we see where we can collaborate.
The TL;DR is that we have built TripleO CLI and Ansible roles to consume Kolla tooling and build our images.
For what its worth, we, the software factory project team, would like to investigate using zuul pipeline to periodically update, test and promote a collection of images. Note that the goal is to update and promote only valid layers (instead of a doing a full rebuild each time). We actually plan to work on that story[0] in the upcoming weeks, it seems like zuul-jobs already feature most of the image building roles we would need, but we might require some modifications to be able to detect if a layer needs to be tested (e.g. looks for "Nothing to do." in stdout) Perhaps we can adapt the zuul-jobs role in such a way that it would support the update use-case as well as using TripleO CLI and roles. Cheers, -Tristan [0] https://tree.taiga.io/project/morucci-software-factory/us/3419
participants (4)
-
Emilien Macchi
-
Monty Taylor
-
Tristan Cacqueray
-
Wesley Hayutin