Open Stack

On 10/8/2015 7:57 PM, Monty Taylor wrote:
> On 10/08/2015 08:39 PM, Robert Collins wrote:
>> This is a bugbear that keeps cropping up and biting us. I'm hoping we
>> can figure out a permanent fix.
>>
>> The problem that occurs is the result of a few interacting things:
>>   - requests has very very specific versions of urllib3 it works with.
>> So specific they aren't always released yet.
>>
>>   - Linux vendors often unbundle urllib3 from requests and then apply
>> what patches were needed to their urllib3; while not updating their
>> requests package dependencies to reflect this.
>>
>>   - we use urllib3 in some places and requests in others (but we don't
>> mix them up)
>>
>>   - if for any reason we have a distro-altered requests + a
>> pip-installed urllib3, requests will [usually] break... see the 'not
>> always released yet' key thing above.
>>
>> Now, there are lots of places this last thing can happen; they all
>> depend on us having a dependency on requests that is compatible with
>> the version installed by the distro, but a urllib3 dependency that
>> triggers an upgrade of just urllib3. When constraints are in use, the
>> requests version has to match the distro requests version exactly, but
>> that will happen from time to time.
>>
>> e.g.
>>
>>   - dvsm test jobs where the base image already has python-requests
>> installed in it
>
> We're working hard to get to the point where this one goes away, fwiw.
>
>>   - virtualenvs where system-site-packages are enabled
>
> These make the easter bunny have sad.
>
>> There are a few strategies that have been proposed to fix this. AIUI
>> they are:
>>   - make sure none of our testing environments include distro requests
>> packages.
>
> yes!
>
>>   - make our requirements be tightly matched to what requests needs to
>> deal with the unbundling
>>
>>   - teach pip how to identify and avoid this situation by always
>> upgrading requests (even if thats just a re-install of the version
>> from PyPI) when the installed requests is a distro installed version
>> **and** urllib3 is being modified.
>>
>>   - get the distros to stop un-vendoring urllib3
>>
>>
>> The first one addresses the situation for the CI gate but doesn't
>> avoid developers getting bitten on their local machines. And
>> installing any distro thing that uses requests would re-instate the
>> potential for breakage. So while its not harmful, I don't think its
>> sufficient to make this go away.
>>
>> The second is trivially insufficient - anytime requests vendored
>> urllib3 is not precisely identical to a released urllib3, it becomes
>> impossible to satisfy that via dependency version pinning - the only
>> way to satisfy it is with the urllib3 in the distro that has whatever
>> change was needed included.
>>
>> The third approach will require some negotiation I suspect - because
>> its aesthetically wrong: from an upstream perspective urllib3 is
>> independent of requests, and vice-versa, but from a distro perspective
>> they are tightly coupled, no variation permitted.
>>
>> The fourth approach meets the stone wall of 'but security' and 'no
>> redundancy permitted' - I don't have the energy to try and get through
>> the near-religious mindset I've encountered there before, though hey -
>> if Fedora and Debian and Ubuntu folk are all interested in figuring
>> out a sustainable way forward, that would be great: please don't feel
>> cut out, I'm just not expecting anything.
>>
>> If there are other approaches, great - please throw them up here.
>
> I've got nothing. I'll continue hacking on #1 just because GATE. But I
> agree, it's necessary but not sufficient.
>
> Thanks for the writeup.
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

FYI, related change that is triggering the conversation:

https://review.openstack.org/#/c/213310/

And there are related bugs in there with more details on other ways this 
fails for people outside the gate system.

-- 

Thanks,

Matt Riedemann