[openstack-dev] [DevStack] Python dependencies: PyPI vs distro packages

Joshua Harlow harlowja at yahoo-inc.com
Tue Aug 6 17:29:35 UTC 2013


Agreed with all your yum issues, its a bit of a pain.

With the help of a few others in anvil we have made a small-mini helper tool (that uses the same underlying yum python apis) so that we can get better control over the whole process.

- https://github.com/stackforge/anvil/tree/master/tools#yyoom
- https://github.com/stackforge/anvil/blob/master/tools/yyoom

The name is meant to be similar for a reason, ha. It is being used to help interact with the yum python api (yes yum is in python) to attempt to overcome some of the issues u stated. Also it has json output which is a lot easier to consume from programs using yyooom (anvil itself), which is much nicer than scraping out the information from the yum output. *note its not really meant to be a full replacement for yum at all, just a helper tool to make the anvil+yum 'pain' a little less* With some tweaking I think it could probably solve your issue of version ranges, since I think the underlying yum library could do it, if the main yum program can't (at this moment).

From: Jay Buffington <me at jaybuff.com<mailto:me at jaybuff.com>>
Reply-To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Tuesday, August 6, 2013 9:40 AM
To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [DevStack] Python dependencies: PyPI vs distro packages

On Tue, Aug 6, 2013 at 8:35 AM, Joshua Harlow <harlowja at yahoo-inc.com<mailto:harlowja at yahoo-inc.com>> wrote:
I think jay your usage also was before anvil started to build all the *missing* dependencies automatically (something u inspired me to get going in the first place) so hopefully said updates to rhel.yaml are only now needed for exceptions and not the common path :)

+1 I'm glad to hear that.  I don't think any of the problems I saw with
Anvil were intractable,  but I do think packaging up the venv is much
less error prone.

My major complaints are issues related to yum. First off, yum can't
resolve a version range (see my message to the yum list:
http://lists.baseurl.org/pipermail/yum/2013-February/023922.html)
which seems like a pretty basic task for a package manager.

Also, Yum seems to do dependency resolution in a single pass.  So
yum can't provide an install solution for cases like this even when a
valid one exists:

package foo depends on pkgdep <= 2.0
package bar depends on pkgdep >= 1.0, <= 2.0
pkgdep 2.1 and 2.0 are available for install

"sudo yum install foo" when none of those three packages are installed
results in an error.  Yum tries to install pkgdep 2.1 and then gives up
when it sees foo doesn't work with pkgdep 2.1 rather than doing a second
pass at building a dependency graph and finding that pkgdep 2.0
would satisfy all dependencies.

Cases like these come up regularly with the 100+ python packages
that openstack depends on (in requirements.txt) when building packages
continuously.  I came to the conclusion that yum is bad software and I
should avoid it globbing up most everything into one big rpm.  Three
months into making the switch I'm happy with that decision.

Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130806/6feef0d0/attachment.html>


More information about the OpenStack-dev mailing list