[openstack-dev] [all][requirements] Why do we use pip install -U as our install_command
tony at bakeyournoodle.com
Thu Feb 11 01:46:53 UTC 2016
I confess up front that I'm pretty green in the is area and there is a lot
of history that I just don't have. That wont stop me from asking/opening the
As I ask in $subject: why do we install with --upgrade in our tox environments?
Are there issues this is fixing/hiding?
I've been seeing a few failures with requirements updates on stable/kilo. I
expect that this applies to liberty and master BUT we're seeing less of it as
constraints are a good thing on those branches.
I'll use glance_store as an example https://review.openstack.org/#/c/265182
I want to be clear this isn't about this specific library (glance_store or
requests), I'm seeing something similar with testtools, fixtures and other libraries.
In py-1.log we (edited for clarity):
pip install --allow-all-external --allow-insecure netaddr -U -rrequirements.txt -rtest-requirements.txt
Collecting python-cinderclient<1.2.0,>=1.1.0 (from -r /home/jenkins/workspace/gate-glance_store-python27/requirements.txt (line 8))
Downloading http://mirror.iad.rax.openstack.org/pypi/packages/py2.py3/p/python-cinderclient/python_cinderclient-1.1.2-py2.py3-none-any.whl (202kB)
Collecting requests!=2.4.0,<2.8.0,>=2.2.0 (from -r /home/jenkins/workspace/gate-glance_store-python27/test-requirements.txt (line 6))
Downloading http://mirror.iad.rax.openstack.org/pypi/packages/2.7/r/requests/requests-2.7.0-py2.py3-none-any.whl (470kB)
So we installed requests 2.7.0 as per our current g-r specification. IIUC We
use the spec from test-requirements as all the requirements+specs from
requirements.txt and test-requirements.txt are processed before looking at the
requirements of each library. So when we look for the requests library while
processing python-cinderclient requirements we already have a spec that's been
satisfied and move on.
Then in py-2.log we (edited for clarity):
pip install --allow-all-external --allow-insecure netaddr -U -e <the git repo>
Requirement already up-to-date: python-cinderclient<1.2.0,>=1.1.0 in ./.tox/py27/lib/python2.7/site-packages (from glance-store==0.4.1.dev16)
Collecting requests!=2.4.0,>=2.2.0 (from python-cinderclient<1.2.0,>=1.1.0->glance-store==0.4.1.dev16)
Downloading http://mirror.iad.rax.openstack.org/pypi/packages/2.7/r/requests/requests-2.9.1-py2.py3-none-any.whl (501kB)
Here we upgrade requests because the python-cinderclient is less restrictive
Here we're only looking at requirements.txt which doesn't have a requests
specification so when we process python-cinderclient's requirements (with -U)
we see a "better" requests library install that and then "go bang" 
I *think* this particular failure would be "fixed" if we didn't install our
packages with -U.
I know that people are working on enhancing the pip dependency resolver but
that isn't work we can use today.
Again there are alternate solutions for this specific issue but I feel like
removing -U would fix a class of problems, perhaps it'll create another I don't
 Footnote deleted in editing and I'm too lazy renumber the rest :D
 Just because it's the one I have open in my browser
 See https://review.openstack.org/#/c/265182
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 473 bytes
Desc: not available
More information about the OpenStack-dev