[ci infra] local PyPi cache/mirroring/proxy
Hello, I don't really have much more than surface experience with our CI infrastructure, but I wanted to start a discussion on a topic that maybe could be picked up by more knowledgeable colleagues. Our tests install a lot of python packages. It almost feels like more than half the time of any test run is spent installing packages. By default, all those packages are being installed from PyPi directly. I wonder if we could gain some advantages by running a local PyPi cache or mirror inside of the CI infrastructure, and making all those tests use it by default for all its package installs. It would: - speed up the downloading of packages, - make tests more resilient in the face of outside network or PyPi outages, - make us better citizens by decreasing the traffic to PyPi and conserving their resources, - give us more control over which packages are available (so, for example, we could easily pin setuptools for all tests, or ban a particular known bad package version, or even do pre-releases of our own packages before we release them on PyPi), Of course nothing is for free, and such a service would require some extra work to set up and maintain. I don't have the experience to be able to tell if the overhead is worth it, but maybe it's something worth considering? Or maybe we are already doing this, and I'm just wasting everyone's time? There is a number of ready to use tools we could utilize for this: https://packaging.python.org/en/latest/guides/index-mirrors-and-caches/#exis... What do you think? -- Radomir Dopieralski
On 13/03/2026 13:20, Radomir Dopieralski wrote:
Hello,
I don't really have much more than surface experience with our CI infrastructure, but I wanted to start a discussion on a topic that maybe could be picked up by more knowledgeable colleagues.
Our tests install a lot of python packages. It almost feels like more than half the time of any test run is spent installing packages. By default, all those packages are being installed from PyPi directly.
I wonder if we could gain some advantages by running a local PyPi cache or mirror inside of the CI infrastructure, and making all those tests use it by default for all its package installs. we already have one each of the ci providrs have a mirror
https://mirror.bhs1.ovh.opendev.org/ with both wheel caches https://mirror.bhs1.ovh.opendev.org/wheel/ and disto cache/mirros our base jobs run https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/configure-mirrors to make sure we are using the provider local mirror which is mange and syced form and afs system the opendeve has run for mnay years at this point.
It would:
- speed up the downloading of packages, - make tests more resilient in the face of outside network or PyPi outages, - make us better citizens by decreasing the traffic to PyPi and conserving their resources, - give us more control over which packages are available (so, for example, we could easily pin setuptools for all tests, or ban a particular known bad package version, or even do pre-releases of our own packages before we release them on PyPi),
Of course nothing is for free, and such a service would require some extra work to set up and maintain. I don't have the experience to be able to tell if the overhead is worth it, but maybe it's something worth considering? Or maybe we are already doing this, and I'm just wasting everyone's time?
There is a number of ready to use tools we could utilize for this: https://packaging.python.org/en/latest/guides/index-mirrors-and-caches/#exis...
What do you think?
-- Radomir Dopieralski
That is great, thank you for the details and sorry for the noise. On Fri, Mar 13, 2026 at 3:21 PM Sean Mooney <smooney@redhat.com> wrote:
On 13/03/2026 13:20, Radomir Dopieralski wrote:
Hello,
I don't really have much more than surface experience with our CI infrastructure, but I wanted to start a discussion on a topic that maybe could be picked up by more knowledgeable colleagues.
Our tests install a lot of python packages. It almost feels like more than half the time of any test run is spent installing packages. By default, all those packages are being installed from PyPi directly.
I wonder if we could gain some advantages by running a local PyPi cache or mirror inside of the CI infrastructure, and making all those tests use it by default for all its package installs. we already have one each of the ci providrs have a mirror
https://mirror.bhs1.ovh.opendev.org/
with both wheel caches https://mirror.bhs1.ovh.opendev.org/wheel/ and disto cache/mirros
our base jobs run
https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/configure-mirrors
to make sure we are using the provider local mirror which is mange and syced form and afs system the opendeve has run for mnay years at this point.
It would:
- speed up the downloading of packages, - make tests more resilient in the face of outside network or PyPi outages, - make us better citizens by decreasing the traffic to PyPi and conserving their resources, - give us more control over which packages are available (so, for example, we could easily pin setuptools for all tests, or ban a particular known bad package version, or even do pre-releases of our own packages before we release them on PyPi),
Of course nothing is for free, and such a service would require some extra work to set up and maintain. I don't have the experience to be able to tell if the overhead is worth it, but maybe it's something worth considering? Or maybe we are already doing this, and I'm just wasting everyone's time?
There is a number of ready to use tools we could utilize for this:
https://packaging.python.org/en/latest/guides/index-mirrors-and-caches/#exis...
What do you think?
-- Radomir Dopieralski
participants (2)
-
Radomir Dopieralski
-
Sean Mooney