On 13/03/2026 13:20, Radomir Dopieralski wrote:
> Hello,
>
> I don't really have much more than surface experience with our CI
> infrastructure, but I wanted to start a discussion on a topic that
> maybe could be picked up by more knowledgeable colleagues.
>
> Our tests install a lot of python packages. It almost feels like more
> than half the time of any test run is spent installing packages. By
> default, all those packages are being installed from PyPi directly.
>
> I wonder if we could gain some advantages by running a local PyPi
> cache or mirror inside of the CI infrastructure, and making all those
> tests use it by default for all its package installs.
we already have one
each of the ci providrs have a mirror

https://mirror.bhs1.ovh.opendev.org/

with both wheel caches https://mirror.bhs1.ovh.opendev.org/wheel/
and disto cache/mirros

our base jobs run
https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/configure-mirrors

to make sure we are using the provider local mirror which is mange and
syced form and afs system the opendeve has run for mnay years at this point.



>  It would:
>
> - speed up the downloading of packages,
> - make tests more resilient in the face of outside network or PyPi
> outages,
> - make us better citizens by decreasing the traffic to PyPi and
> conserving their resources,
> - give us more control over which packages are available (so, for
> example, we could easily pin setuptools for all tests, or ban a
> particular known bad package version, or even do pre-releases of our
> own packages before we release them on PyPi),
>
> Of course nothing is for free, and such a service would require some
> extra work to set up and maintain. I don't have the experience to be
> able to tell if the overhead is worth it, but maybe it's something
> worth considering? Or maybe we are already doing this, and I'm just
> wasting everyone's time?
>
> There is a number of ready to use tools we could utilize for this:
> https://packaging.python.org/en/latest/guides/index-mirrors-and-caches/#existing-projects
>
> What do you think?
>
> -- Radomir Dopieralski