[openstack-dev] [DevStack] Python dependencies: PyPI vs distro packages

Dean Troyer dtroyer at gmail.com
Tue Aug 6 15:19:10 UTC 2013


On Mon, Aug 5, 2013 at 8:37 PM, Ian Wienand <iwienand at redhat.com> wrote:
> I think Anvil is working with the package management system so that
> scenario doesn't happen.  The "fine, those can be re-installed with
> pip" bit is where the problem occurs.

Agreed.

> The "whole lot" bit is important, because you can't have conflicts
> there.  Say requirements.txt brings in setuptools-0.9.8; Anvil will
> create a python-setuptools 0.9.8 package.  If rpm-packaged nose relies
> *exactly* python-setuptools at 0.6.10, there will be a conflict -- I
> think the installation would fail to complete.  But likely, that
> package doesn't care and gets it dep satisfied by 0.9.8 [1]

How many times has that bitten us already just in the PyPI world?

> Because you're not using pip to install directly, you don't have this
> confusion around who owns files in /usr/lib/python2.6/site-packages
> and have rpm or pip overwriting each other -- RPM owns all the files
> and that's that.

I agree that is the way the world is supposed to work.  I wore the
sysadmin hat for far longer than I care to admit and spent a lot of
time building packages to fix things like this.  That's easy for stuff
that is released every 1-2 years.  For the rate of churn we have it is
a treadmill.

>> Removing python-setuptools will also remove python-nose, numpy and
>> other packages depending on what is installed.
>
> Nowhere is python-setuptools removed; just upgraded.

That example refers to the current path of removing python-setuptools
and replacing it with a version installed from source.

> [1] From a quick look at Anvil I don't think it would handle this
> situation, which is probably unsolvable (if a rpm-package wants one
> version and devstack wants another, and it all has to be in
> /usr/lib/python2.6/site-packages, then *someone* is going to lose).

And that is the crux of the problem.  When both can be installed
side-by-side and sys.path is in control of the order, things work.
This is such a fundamental problem in the distro that I am beginning
to thing that we need to address it ourselves.  Everything else is
treating symptoms.

dt

-- 

Dean Troyer
dtroyer at gmail.com



More information about the OpenStack-dev mailing list