Open Stack

On 2020-01-13 14:43:56 +0000 (+0000), Jeremy Stanley wrote:
> On 2020-01-13 15:16:58 +0100 (+0100), Radosław Piliszek wrote:
> > It turned out to be real mess to fix devstack and devstack-gate
> > and whatever else could be impacted. The current goal is to get
> > rid of setuptools wheel so that it does not interfere in Zuul. As
> > more packages go py3-only we will be forced to generate proper
> > metadata in HTML indices...
> [...]
> 
> Yes, pardon me as I'm still catching up. Based on what I've read so
> far, the PyPI simple API provides some additional information with
> its package indices indicating which Python releases are supported
> by a given package. Our wheel cache is just being served from Apache
> with mod_autoindex providing basic indexing of the files, so does
> not have that information to provide.
> 
> We've discussed a number of possible solutions to the problem. One
> option is to (temporarily) stop using the pre-built wheel cache, but
> that presupposes that it doesn't do much for us in the first place.
> That cache is there to provide pre-built wheels for packages which
> otherwise take a *very* long time to build from sdist and for which
> there is on usable wheel on PyPI (at least for the platforms on
> which we're running our jobs).

This was proposed via https://review.opendev.org/702166 for the
record.

> Another option is to stop unnecessarily copying wheels already
> available on PyPI into that cache. It's used as a sieve, so that
> jobs can pull wheels from it when they exist but will still fall
> back to PyPI for any the cache doesn't contain. I suspect that the
> majority of projects dropping compatibility with older Python
> releases publish usable wheels for our platforms on PyPI already, so
> their presence in our cache is redundant. We could remove the latest
> Setuptools release from the wheel-mirror volume as a short-term
> solution, but will need to temporarily stop further updates to the
> cache since the job which builds it would just put that file right
> back.

I've proposed https://review.opendev.org/702244 for a smaller-scale
version of this now (explicitly blacklisting wheels for pip,
setuptools and virtualenv) but we can expand it to something more
thorough once it's put through its paces. If we merge that, then we
can manually delete the affected setuptools wheel from our
wheel-mirror volume and not have to worry about it coming back.
-- 
Jeremy Stanley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200113/98c840e0/attachment.sig>