[Kolla][Kolla-Ansible] Making the use of mirrors easier on container build
Oh hello, For the last couple of years I've been working on trying to improve the VDI experience in OpenStack by enriching our SPICE capabilities. A series of patches has now landed in Nova that enable SPICE native console sessions from outside the cloud, I have a working (but probably not completely finished) SPICE native proxy, and the next step is adding deployment support to Kolla-Ansible so that I can start demoing an end to end implementation. I have a draft of those changes, but don't feel quite ready to push them to Gerrit just yet. As part of that I've been building my own Kolla images in my homelab CI environment for a couple of years now. I don't know about others, but for me by far the most common reason for these builds to fail is because of intermittent connectivity to upstream package repositories -- either flaky networking, random noise, or rate limiting. Increasingly I find myself carrying private patches to use local mirrors instead of the upstream sources. It occurs to me I might not be the only one with this sort of use case. I therefore wonder how the Kolla team would feel about abstracting the upstream sources of various build dependencies so that they were defined centrally? That would make it much easier both to override sources with local mirrors, as well as define exactly what upstream sources are required to conduct a build. If that's a reasonable thing to do, where would people see that list sitting? My initial thought is in the defaults for the kolla-build ansible role, but I am definitely not a Kolla expert so I am open to guidance here. Cheers, Michael -- Person on the internet. https://madebymikal.com. He / his / him.
Hi Michael, The sources are defined centrally - distribution repositories are in repos.yaml - https://opendev.org/openstack/kolla/src/branch/master/kolla/template/repos.y... The per image tarballs that we fetch from different sources are overridable via kolla-build.conf - see https://docs.openstack.org/kolla/latest/admin/image-building.html#python-pac... Is there anything we’re missing? Michał Nasiadka mnasiadka@gmail.com W dniu sob., 4 paź 2025 o 09:08 Michael Still <mikal@stillhq.com> napisał(a):
Oh hello,
For the last couple of years I've been working on trying to improve the VDI experience in OpenStack by enriching our SPICE capabilities. A series of patches has now landed in Nova that enable SPICE native console sessions from outside the cloud, I have a working (but probably not completely finished) SPICE native proxy, and the next step is adding deployment support to Kolla-Ansible so that I can start demoing an end to end implementation. I have a draft of those changes, but don't feel quite ready to push them to Gerrit just yet.
As part of that I've been building my own Kolla images in my homelab CI environment for a couple of years now. I don't know about others, but for me by far the most common reason for these builds to fail is because of intermittent connectivity to upstream package repositories -- either flaky networking, random noise, or rate limiting. Increasingly I find myself carrying private patches to use local mirrors instead of the upstream sources.
It occurs to me I might not be the only one with this sort of use case. I therefore wonder how the Kolla team would feel about abstracting the upstream sources of various build dependencies so that they were defined centrally? That would make it much easier both to override sources with local mirrors, as well as define exactly what upstream sources are required to conduct a build.
If that's a reasonable thing to do, where would people see that list sitting? My initial thought is in the defaults for the kolla-build ansible role, but I am definitely not a Kolla expert so I am open to guidance here.
Cheers, Michael
-- Person on the internet. https://madebymikal.com. He / his / him.
On Sun, Oct 5, 2025 at 3:12 AM Michał Nasiadka <mnasiadka@gmail.com> wrote:
Hi Michael,
The sources are defined centrally - distribution repositories are in repos.yaml -
https://opendev.org/openstack/kolla/src/branch/master/kolla/template/repos.y...
The per image tarballs that we fetch from different sources are overridable via kolla-build.conf - see
https://docs.openstack.org/kolla/latest/admin/image-building.html#python-pac...
Is there anything we’re missing?
Ahh, I had somehow missed that. The specific example which causes me pain at the moment is the OSBPO debian mirror. For reasons I cannot explain I regularly get at 5XX HTTP error from that one. I am therefore carrying this patch: diff --git a/docker/base/Dockerfile.j2 b/docker/base/Dockerfile.j2 index 638aa8a..ce6fcf4 100644 --- a/docker/base/Dockerfile.j2 +++ b/docker/base/Dockerfile.j2 @@ -262,7 +262,7 @@ COPY sources.list /etc/apt/sources.list.d/kolla-custom.list RUN apt update \ && apt install -y --no-install-recommends extrepo \ - && extrepo enable openstack_epoxy \ + && extrepo enable openstack_epoxy --mirror http://mirror-osbpo.home.stillhq.com/debian \ && apt purge -y extrepo \ && apt --purge autoremove -y \ && apt clean So perhaps this is more an example of a special case being missed? Thanks, Michael
participants (2)
-
Michael Still
-
Michał Nasiadka