[openstack-dev] [all][requirements] Why do we use pip install -U as our install_command
cboylan at sapwetik.org
Thu Feb 11 03:09:54 UTC 2016
On Wed, Feb 10, 2016, at 05:46 PM, Tony Breeds wrote:
> Hi All,
> I confess up front that I'm pretty green in the is area and there is
> a lot
> of history that I just don't have. That wont stop me from asking/opening
> As I ask in $subject: why do we install with --upgrade in our tox
> Are there issues this is fixing/hiding?
> I've been seeing a few failures with requirements updates on stable/kilo.
> expect that this applies to liberty and master BUT we're seeing less of
> it as
> constraints are a good thing on those branches.
> I'll use glance_store as an example
> I want to be clear this isn't about this specific library (glance_store
> requests), I'm seeing something similar with testtools, fixtures and
> other libraries.
> Looking at:
> In py-1.log we (edited for clarity):
> pip install --allow-all-external --allow-insecure netaddr -U
> -rrequirements.txt -rtest-requirements.txt
> Collecting python-cinderclient<1.2.0,>=1.1.0 (from -r
> /home/jenkins/workspace/gate-glance_store-python27/requirements.txt (line
> Collecting requests!=2.4.0,<2.8.0,>=2.2.0 (from -r
> (line 6))
> So we installed requests 2.7.0 as per our current g-r specification.
> IIUC We
> use the spec from test-requirements as all the requirements+specs from
> requirements.txt and test-requirements.txt are processed before looking
> at the
> requirements of each library. So when we look for the requests library
> processing python-cinderclient requirements we already have a spec that's
> satisfied and move on.
> Then in py-2.log we (edited for clarity):
> pip install --allow-all-external --allow-insecure netaddr -U -e <the git
> Requirement already up-to-date: python-cinderclient<1.2.0,>=1.1.0 in
> ./.tox/py27/lib/python2.7/site-packages (from glance-store==0.4.1.dev16)
> Collecting requests!=2.4.0,>=2.2.0 (from
> Here we upgrade requests because the python-cinderclient is less
> Here we're only looking at requirements.txt which doesn't have a requests
> specification so when we process python-cinderclient's requirements (with
> we see a "better" requests library install that and then "go bang" 
> I *think* this particular failure would be "fixed" if we didn't install
> packages with -U.
> I know that people are working on enhancing the pip dependency resolver
> that isn't work we can use today.
> Again there are alternate solutions for this specific issue but I feel
> removing -U would fix a class of problems, perhaps it'll create another I
> Discuss :)
> Yours Tony.
>  Footnote deleted in editing and I'm too lazy renumber the rest :D
>  Just because it's the one I have open in my browser
>  See https://review.openstack.org/#/c/265182
The reason that I remember off the top of my head is because we spent
far too much time telling people to run `tox -r` when their code failed
during Jenkins testing but ran just fine locally. It removes a
significant amount of debugging overhead to have everyone using a
relatively consistent set of packages whenever they rerun tests.
More information about the OpenStack-dev