Open Stack

Excerpts from Ian Cordasco's message of 2014-09-18 10:33:04 -0700:
> 
> On 9/18/14, 11:29 AM, "Clint Byrum" <clint at fewbar.com> wrote:
> 
> >Excerpts from Donald Stufft's message of 2014-09-18 07:30:27 -0700:
> >> 
> >> > On Sep 18, 2014, at 10:18 AM, Clint Byrum <clint at fewbar.com> wrote:
> >> > 
> >> > Excerpts from Donald Stufft's message of 2014-09-18 04:58:06 -0700:
> >> >> 
> >> >>> On Sep 18, 2014, at 7:54 AM, Thomas Goirand <zigo at debian.org> wrote:
> >> >>> 
> >> >>>> 
> >> >>>> Linux distributions are not the end be all of distribution models
> >>and
> >> >>>> they don’t get to dictate to upstream.
> >> >>> 
> >> >>> Well, distributions is where the final user is, and where software
> >>gets
> >> >>> consumed. Our priority should be the end users.
> >> >> 
> >> >> 
> >> >> Distributions are not the only place that people get their software
> >>from,
> >> >> unless you think that the ~3 million downloads requests has received
> >> >> on PyPI in the last 30 days are distributions downloading requests to
> >> >> package in their OSs.
> >> >> 
> >> > 
> >> > Do pypi users not also need to be able to detect and fix any versions
> >> > of libraries they might have? If one has some virtualenvs with various
> >> > libraries and apps installed and no --system-site-packages, one would
> >> > probably still want to run 'pip freeze' in all of them and find out
> >>what
> >> > libraries are there and need to be fixed.
> >> > 
> >> > Anyway, generally security updates require a comprehensive strategy.
> >> > One common comprehensive strategy is version assertion.
> >> > 
> >> > Vendoring complicates that immensely.
> >> 
> >> It doesn’t really matter. PyPI doesn’t dictate to projects who host
> >>there what
> >> that project is allowed to do except in some very broad circumstances.
> >>Whether
> >> or not requests *should* do this doesn't really have any bearing on what
> >> Openstack should do to cope with it. The facts are that requests does
> >>it, and
> >> that people pulling things from PyPI is an actual platform that needs
> >>thought
> >> about.
> >> 
> >> This leaves Openstack with a few reasonable/sane options:
> >> 
> >> 1) Decide that vendoring in requests is unacceptable to what Openstack
> >>as a
> >>    project is willing to support, and cease the use of requests.
> >> 2) Decide that what requests offers is good enough that it outweighs
> >>the fact
> >>    that it vendors urllib3 and continue using it.
> >> 
> >
> >There's also 3) fork requests, which is the democratic way to vote out
> >an upstream that isn't supporting the needs of the masses.
> 
> Given requests’ download count, I have to doubt that OpenStack users
> constitute the masses in this case.
> 

This wasn't "the masses" from the requests stand point, but from the
OpenStack standpoint. Consider the case of a small island territory
of a much larger nation. At some point most of them have claimed their
independence from the larger nation unless the larger nation is willing
to step up and make them a full member with a real vote. This allows
them to act in their best interest. So even if it means a much more
difficult road, it is the road most advantageous to them.

Also upon reflection, it's a bit interesting that forking requests is
being dismissed so quickly, when in essence, requests maintains a fork
of urllib3 in tree (albeit, one that is just a fork from the _releases_,
not from the whole project).

> >I don't think we're anywhere near there, but I wanted to make it clear
> >there _is_ a more extreme option.
> >
> >> If the 2nd option is chosen, then doing anything but supporting the
> >>fact that
> >> requests vendors urllib3 within the code that openstack writes is
> >>hurting the
> >> users who fetch these projects from PyPI because you don't agree with
> >>one of
> >> the choices that requests makes. By all means do conditional imports to
> >>lessen
> >> the impact that the choice requests has made (and the one that
> >>Openstack has
> >> made to use requests) on downstream distributors, but unconditionally
> >>importing
> >> from the top level urllib3 for use within requests is flat out wrong.
> >> 
> >> Obviously neither of these options excludes the choice to lean on
> >>requests to
> >> reverse this decision as well. However that is best done elsewhere as
> >>the
> >> person making that decision isn't a member of these mailing lists as
> >>far as
> >> I am aware.
> >> 
> >
> >To be clear, I think we should keep using requests. But we should lend
> >our influence upstream and explain that our users are required to deal
> >with this in a way that perhaps hasn't been considered or given the
> >appropriate priority.
> 
> It’s been considered several times. There have been multiple issues.
> There’s more than just the one you linked to. The decision is highly
> unlikely to change whether it’s coming from a group of people in OpenStack
> or another distribution package maintainer.
> 

Indeed, hence my thinking that forking requests might be in order. Even
if that fork is just a long lived fork that stays mostly in sync, but
without urllib3 vendored. I think that has actually already happened in
the distros... so I wonder how painful it would be to do the same thing
on pypi, and let the distros just consume that.

Anyway, I'm not going to take that challenge on, so I think this
discussion should end here. We'll just have to let the distros keep doing
the job of supporting this form of risk assessment. The rest of us are
just going to have to keep trying the actual exploit test code everywhere.