[openstack-dev] Please do *NOT* use "vendorized" versions of anything (here: glanceclient using requests.packages.urllib3)

Donald Stufft donald at stufft.io
Thu Sep 18 02:43:12 UTC 2014


> On Sep 17, 2014, at 10:24 PM, Thomas Goirand <zigo at debian.org> wrote:
> 
> On 09/18/2014 08:22 AM, Morgan Fainberg wrote:
>> I think that all of the conversation to this point has been valuable,
>> the general consensus is vendoring a library is not as desirable as
>> using it strictly as a dependency. It would be nice in a perfect
>> world if vendoring wasn’t and issue, but in this case I think the
>> root of the matter is that Debian un-vendors urllib3 and we have
>> referenced the vendored urllib3 instead of installing and utilizing
>> urllib3 directly.
>> 
>> This poses at least one problem for us: we are not able to guarantee
>> we’re using the same urllib3 library as requests is. I am unsure how
>> big of a deal this ends up being, but it is a concern and has brought
>> up a question of how to handle this in the most appropriate and
>> consistent way across all of the distributions we as OpenStack support. 
>> 
>> Does this make requests a bad library we should toss aside for
>> something else? Instead of being concerned with the reasons for
>> vendoring urllib3 (or un-vendoring it) we should shift the conversation
>> towards two questions:
>> 
>> 1. Is it a real issue if the version of urllib3 is mismatched between
>> our client libraries and requests? 
>> 2. If it is a real issue how are we solving it?
> 
> The main issue is that urllib3 in requests, as other pointed out, is not
> up-to-date, and will not be updated. In fact, that's the main reason why
> the upstream authors of requests are vendorizing: it's because they
> don't want to carry the burden of staying up-to-date.

I don’t think this is remotely true, often times requests updates itself
to versions of urllib3 which aren’t even released yet. Sometimes urllib3
might make commits and do a release that happens between requests
versions though. I mean technically they might be not up to date until
their next version release though.

> 
> And then, there's incompatibilities and divergences that appear, leading
> to all sorts of unexpected issues, like one thing working with pip, but
> not with the packages. This kind of issues are very hard to understand
> and debug. Distributions may report the issue upstream, then upstream
> will say "but it's working for me", and then we may loose a lot of time.
> This happened already, and may happen again if we don't care enough.

I think this is bound to happen anytime you have downstream modifying
things. It happens in pip (pip vendors things too) and yea it’s quite annoying
but part of PEP 440 is providing ways for downstream to signal they’ve
modified things so that instead of “foo 1.0” you have “foo 1.0+ubuntu1” or
whatever.

> 
>> Obviously we can work with the requests team to figure out the best
>> approach.
> 
> There's only a single approach that works: have the requests upstream
> authors to stop embedding foreign code, and use the dependency instead.

There are legitimate reasons for a project to vendor things. Linux distributions
are not the end be all of distribution models and they don’t get to dictate to
upstream.

Generally I agree that requests should not vendor urllib3, but it’s not a clear
cut thing where there is one right way to do it.

> 
>> We should focus on the solution here rather than continuing
>> down the path of whether requests should/shouldn’t be vendoring it’s
>> dependencies since it is clear that the team has their reasons and
>> does not want to switch to the dependency model again.
> 
> I'm sure they have tons of wrong reasons. If they don't want to change
> anything, then we can only try to work around the issue, and never use
> the embedded version.

Generally you either work with the embedded versions or you don’t
use requests. You’re going to get very strange incompatibility problems
if you try to mis requests.packages.urllib3 and urllib3 in one codebase
and if you’re using requests at all it’s going to be expecting to use
the embedded copy of urllib3.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140917/9f0ad045/attachment.html>


More information about the OpenStack-dev mailing list