On 2020-01-13 15:16:58 +0100 (+0100), Radosław Piliszek wrote:
It turned out to be real mess to fix devstack and devstack-gate and whatever else could be impacted. The current goal is to get rid of setuptools wheel so that it does not interfere in Zuul. As more packages go py3-only we will be forced to generate proper metadata in HTML indices... [...]
Yes, pardon me as I'm still catching up. Based on what I've read so far, the PyPI simple API provides some additional information with its package indices indicating which Python releases are supported by a given package. Our wheel cache is just being served from Apache with mod_autoindex providing basic indexing of the files, so does not have that information to provide. We've discussed a number of possible solutions to the problem. One option is to (temporarily) stop using the pre-built wheel cache, but that presupposes that it doesn't do much for us in the first place. That cache is there to provide pre-built wheels for packages which otherwise take a *very* long time to build from sdist and for which there is on usable wheel on PyPI (at least for the platforms on which we're running our jobs). Another option is to stop unnecessarily copying wheels already available on PyPI into that cache. It's used as a sieve, so that jobs can pull wheels from it when they exist but will still fall back to PyPI for any the cache doesn't contain. I suspect that the majority of projects dropping compatibility with older Python releases publish usable wheels for our platforms on PyPI already, so their presence in our cache is redundant. We could remove the latest Setuptools release from the wheel-mirror volume as a short-term solution, but will need to temporarily stop further updates to the cache since the job which builds it would just put that file right back. -- Jeremy Stanley