Open Stack

Excerpts from Clay Gerrard's message of 2015-07-28 19:25:04 -0700:
> Doug,
> 
> I believe our glance friends are not the only project with some open
> questions on dealing with the "required dependency for optional plugin"
> use-case.  You've made a recommendation to leverage some python tooling
> functionality that I'm not familiar with.  I was hoping I could probe you
> to elaborate so I can try and educate myself more?
> 
> ... inline
> 
> On Tue, Jul 28, 2015 at 4:55 PM, Doug Hellmann <doug at doughellmann.com>
> wrote:
> 
> >
> >
> > Please set up an "extras" entry for each backend instead of just
> > removing the dependencies.  That will signal to users that you know
> > what dependencies there are for a backend,
> 
> 
> You referenced nova [1], and oslo.versionedobjects [2] for examples - but
> I'd be more curious for the documentation if you have any idea where I
> might look for it?  Is this a feature of pkg_resources, distutils,
> setuptools, pbr?  What exactly does describing dependencies via this
> "extras" key afford?

There is some documentation in the pbr manual
(http://docs.openstack.org/developer/pbr/#extra-requirements). The
feature is implemented throughout the packaging tool chain now.

"Extras" is a way to provide, well, extra dependencies for cases
where some items are optional. We can use it for "test" related
items in libraries, so that for example the fixtures package is not
dragged into production deployments. We can also use it in the case
here in glance_store and in oslo.messaging, where a particular
driver needs supporting libraries that are only needed if that
driver is being used.

Extras are a useful way to signal these sorts of dependencies for
anyone installing via pip (which understands them) but also to
downstream packagers.

> 
> > but that they are optional,
> > and still allow someone to do the equivalent of "pip install
> > glance[vmware]" or "pip install glance[swift]" to get those
> > dependencies.
> 
> 
> I'm not familiar with that syntax for pip or it's equivalent!  That sounds
> awesome!  Can you do like [extras:<pluginname>] in your setup.cfg and pip
> install project[<pluginname>] just works!?  OMGBBQ!

The extras are listed under the "extras" section in setup.cfg. Each
one has a name, and a list of the package specifications.  The
example from the pbr guide is:

[extras]
security =
    aleph
    bet :python_environment=='3.2'
    gimel :python_environment=='2.7'
testing =
    quux :python_environment=='2.7'

Here one would say "pip install foo[testing]" and on Python 2.7
would also get the quux library.

> 
> > Nova and oslo.versionedobjects have examples in their
> > setup.cfg if you need a template.
> >
> 
> Hrm... I'm missing how either one of those setup.cfg's [1, 2] include an
> example relevant to this use-case (i.e. required dependency for optional
> backend plugin)?

Oh, bother, that was a grep gone awry. Here's the example from
oslo.versionedobjects:

[extras]
fixtures =
  mock>=1.2
  fixtures>=1.3.1

If you're writing tests using oslo.versionedobjects in your project,
you would add "oslo.versionedobjects[fixtures]" to your
test-requirements.txt to get the supporting libraries needed by the
fixtures module in the library.

> 
> >
> > I didn't mention in the reviews, but this will also make integration
> > tests in our gate easier, since you can put ".[vmware]" or ".[swift]" in
> > the tox.ini to pull in those dependencies.
> >
> >
> Hrm... yes testing.  So that's part "just a new -e" for the tox.ini - but
> I'm not quite sure I follow how each environment would specify different
> dependencies for the virtualenv?

Set different values for the "deps" variable in the tox.ini.

> 
> I hope you can point me to some more information on the subject.
> 
> Thank you very much for pushing this out to a wider audience,
> 
> clayg
> 
> 1. https://github.com/openstack/nova/blob/master/setup.cfg
> 2.
> https://github.com/openstack/oslo.versionedobjects/blob/master/setup.cfg#L25