[openstack-dev] how to provide tests environments for python things that require C extensions

Yuriy Taraday yorik.sar at gmail.com
Wed Sep 10 20:34:56 UTC 2014


On Tue, Sep 9, 2014 at 9:58 PM, Doug Hellmann <doug at doughellmann.com> wrote:

>
> On Sep 9, 2014, at 10:51 AM, Sean Dague <sean at dague.net> wrote:
>
> > On 09/09/2014 10:41 AM, Doug Hellmann wrote:
> >>
> >> On Sep 8, 2014, at 8:18 PM, James E. Blair <corvus at inaugust.com> wrote:
> >>
> >>> Sean Dague <sean at dague.net> writes:
> >>>
> >>>> The crux of the issue is that zookeeper python modules are C
> extensions.
> >>>> So you have to either install from packages (which we don't do in unit
> >>>> tests) or install from pip, which means forcing zookeeper dev packages
> >>>> locally. Realistically this is the same issue we end up with for mysql
> >>>> and pg, but given their wider usage we just forced that pain on
> developers.
> >>> ...
> >>>> Which feels like we need some decoupling on our requirements vs. tox
> >>>> targets to get there. CC to Monty and Clark as our super awesome tox
> >>>> hackers to help figure out if there is a path forward here that makes
> sense.
> >>>
> >>> From a technical standpoint, all we need to do to make this work is to
> >>> add the zookeeper python client bindings to (test-)requirements.txt.
> >>> But as you point out, that makes it more difficult for developers who
> >>> want to run unit tests locally without having the requisite libraries
> >>> and header files installed.
> >>
> >> I don’t think I’ve ever tried to run any of our unit tests on a box
> where I hadn’t also previously run devstack to install all of those sorts
> of dependencies. Is that unusual?
> >
> > It is for Linux users, running local unit tests is the norm for me.
>
> To be clear, I run the tests on the same host where I ran devstack, not in
> a VM. I just use devstack as a way to bootstrap all of the libraries needed
> for the unit test dependencies. I guess I’m just being lazy. :-)


You can't run devstack everywhere you code (and want to run tests). I, for
example, can't run devstack on my work laptop because I use Gentoo there.
And I have MacOS X on my home laptop, so no devstack there too. The latter
should be more frequent case in the community.

That said I never had a problem with emerging (on either of systems)
necessary C libraries for tests to run. As long as they don't pull a lot of
(or any) Linux-specific dependencies, it's fine.

For me this issue is the case for setuptools' extras. The only problem with
them is that we can't specify them in requirement.txt files currently, so
we'd have to add another hack to pbr to gather extra dependencies from
files like requirements-<extra_name>.txt or smth like that.
Then we can provide different tox venvs for diferent extras sets.

-- 

Kind regards, Yuriy.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140911/ad5995d2/attachment.html>


More information about the OpenStack-dev mailing list