[openstack-dev] [All]Optional dependencies and requirements.txt

Ben Nemec openstack at nemebean.com
Mon Feb 17 22:57:43 UTC 2014


On 2014-02-12 21:35, David Koo wrote:
>> Most of the reason for the #2 there is for plugins that taskflow can
>> use (but which are not required for all users). Ideally there would be
>> more dynamic way to list requirements of libraries and projects, one
>> that actually changes depends on the features used/desired to be used.
> 
> Disclaimer: Not a pip/setuptools developer - only a user.
> 
> From the pip & setuptools docs, the "extras" feature[1] of these tools
> seems to do what you want:
> 
>     SQLAlchemy<=0.7.99,<=0.9.99 [SQLAlchemyPersistence]
>     alembic>=0.4.1 [SQLAlchemyPersistence]
>     ...
> 
> [1] http://www.pip-installer.org/en/1.1/requirements.html

That does sound interesting.  It could allow us to have oslo.messaging 
handle all of our messaging requirements rather than having them 
included directly in each project like they are today.  You could do 
things like

oslo.messaging[rabbitmq]

or

oslo.messaging[qpid]

to install the appropriate deps through oslo.messaging.

Unfortunately, it doesn't solve the immediate problem of specifying 
those things in the consuming projects.  With the above example, you 
would still need to have separate requirements files for rabbitmq and 
qpid in the project using oslo.messaging.

Still, it's something that would be nice to make use of at some point, 
if it will play nicely with our setup.py.

-Ben

> 
> --
> Koo
> 
> On Thu, 13 Feb 2014 02:37:44 +0000
> Joshua Harlow <harlowja at yahoo-inc.com> wrote:
> 
>> In taskflow we've done something like the following.
>> 
>> Its not perfect, but it is what currently works (willing to take
>> suggestions on better).
>> 
>> We have three different requirements files.
>> 
>> 1. Required to work in any manner @
>> https://github.com/openstack/taskflow/blob/master/requirements.txt 2.
>> Optionally required to work (depending on features used) @
>> https://github.com/openstack/taskflow/blob/master/optional-requirements.txt
>> 3. Test requirements (only for testing) @
>> https://github.com/openstack/taskflow/blob/master/test-requirements.txt
>> 
>> Most of the reason for the #2 there is for plugins that taskflow can
>> use (but which are not required for all users). Ideally there would
>> be more dynamic way to list requirements of libraries and projects,
>> one that actually changes depends on the features used/desired to be
>> used. In a way u could imagine a function taking in a list of desired
>> features (qpid vs rabbit could be one such feature) and that function
>> would return a list of requirements to work with this feature set.
>> Splitting it up into these separate files is doing something similar
>> (except using static files instead of just a function to determine
>> this). I'm not sure if anything existing (or is planned) for making
>> this situation better in python (it would be nice if there was a way).
>> 
>> -Josh
>> 
>> From: Doug Hellmann
>> <doug.hellmann at dreamhost.com<mailto:doug.hellmann at dreamhost.com>>
>> Reply-To: "OpenStack Development Mailing List (not for usage
>> questions)"
>> <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
>> Date: Wednesday, February 12, 2014 at 1:42 PM To: Ben Nemec
>> <openstack at nemebean.com<mailto:openstack at nemebean.com>>, "OpenStack
>> Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
>> Subject: Re: [openstack-dev] [All]Optional dependencies and
>> requirements.txt
>> 
>> 
>> 
>> 
>> On Wed, Feb 12, 2014 at 3:58 PM, Ben Nemec
>> <openstack at nemebean.com<mailto:openstack at nemebean.com>> wrote: Hi all,
>> 
>> This is an issue that has come up recently in tripleo as we try to
>> support more varied configurations.  Currently qpid-python is not
>> listed in requirements.txt for many of the OpenStack projects, even
>> though they support using Qpid as a messaging broker.  This means
>> that when we install from source in tripleo we have to dynamically
>> add a line to requirements.txt if we want to use Qpid (we pip install
>> -r to handle deps).  There seems to be disagreement over the correct
>> way to handle this, so Joe requested on my proposed Nova change that
>> I raise the issue here.
>> 
>> There's already some discussion on the bug here:
>> https://bugs.launchpad.net/heat/+bug/1225191 as well as a separate
>> Neutron bug here: https://bugs.launchpad.net/neutron/+bug/1225232
>> 
>> If there's a better alternative to "require all the things" I'm
>> certainly interested to hear it.  I expect we're going to hit this
>> more in the future as we add support for other optional backends for
>> services and such.
>> 
>> We could use a separate requirements file for each driver, following
>> a naming convention to let installation tools pick up the right file.
>> For example, oslo.messaging might include amqp-requirements.txt,
>> qpid-requirements.txt, zmq-requirements.txt, etc.
>> 
>> That would complicate the global requirements sync script, but not
>> terribly so.
>> 
>> Thoughts?
>> 
>> Doug
>> 
>> 
>> 
>> Thanks.
>> 
>> -Ben
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list