[openstack-dev] Requests + urllib3 + distro packages

Joshua Harlow harlowja at fastmail.com
Mon Oct 12 22:44:47 UTC 2015

Jeremy Stanley wrote:
> On 2015-10-12 15:40:48 +0200 (+0200), Thomas Goirand wrote:
> [...]
>> Has the infra team ever thought about doing that for (at least) all of
>> the 3rd party libs we use? I'd love to work closer with the infra team
>> to provide them with missing packages they would need, and I'm sure my
>> RPM buddy Haikel would too. This also would help getting our
>> openstack/{deb,rpm}- projects up to speed as well.
> [...]
> Long ago there was an idea that we might somehow be able to
> near-instantly package anything on PyPI and serve RPMs and DEBs of
> it up to CI jobs, but doing that would be a pretty massive (and in
> my opinion very error-prone) undertaking.

Anvil gets somewhat far on this, although its not supporting DEBs it 
does build its best attempt at RPMs building them automatically and 
turning git repos of projects into RPMs.

http://anvil.readthedocs.org/en/latest/topics/summary.html (hopefully 
the existence of this is not news to folks).

A log of this in action (very verbose) is at:


Feel free to ask questions as to how this is being done on 
#openstack-anvil or find me on other channels... I believe it is also 
one of the goals of the rpm and packaging teams to do something similar, 
although that might be a ways off?

> Right now we can take advantage of the fact that the Python
> ecosystem uploads new releases to PyPI as their primary distribution
> channel. By using pip to install dependencies from PyPI in our CI
> system, we can pretty instantly test compatibility of our software
> with new releases of dependencies (much faster that they can get
> properly packaged in distros), and easily test different versions by
> proposing changes to the openstack/requirements repository.
> The only way I see this changing is if authors of Python libraries
> switch to packaging their own software for major distributions and
> have a pass to get them included by those distributions
> near-instantly. Also, distros would have to cease caring about
> reducing the number of concurrent versions of libraries they provide
> (and I posit that as soon as debian/sid ships a DEB for every
> version ever released for packages like python-requests and
> python-urllib3, apt-get will begin to exhibit similar dependency
> resolution challenges).

More information about the OpenStack-dev mailing list