Open Stack

Hi Victor,

This page is worth it : https://wiki.openstack.org/wiki/Testr
Comments inline.

-Sylvain

2014-04-08 10:13 GMT+02:00 victor stinner <victor.stinner at enovance.com>:

> Hi,
>
> I have some issues when running unit tests in OpenStack. I would like to
> help, but I don't know where I should start and how I can fix these bugs.
> My use case is to run unit tests and rerun a single test if one or more
> tests failed. Well, it should be the most basic use case, no?
>
>
> (1) First problem: if a Python module cannot be loaded, my terminal is
> flooded with a binary stream which looks like:
>
> ... tCase.test_deprecated_without_replacement\xd7\xe1\x06\xa1\xb3)\x01 at l@atests.unit.test_versionutils.DeprecatedTestCa
> ...
>
> IMO it's a huge bug in the testr tool: "testr run" command should not
> write binary data into stdout. It makes development very hard.
>
>

That's happening when testr is trying to locate all unittests classes.
Switching to nosetests without parrallelism can help, as the traceback is
shown there.


>
> (2) When a test fails, it's hard to find the command to rerun a single
> failing test.
>
> Using the tool trial, I can just copy/paste the "FQDN" name of the failing
> test and run "trial FQDN". Example:
>
>    trial tests.unit.test_timeutils.TestIso8601Time.test_west_normalize
>
> Using the tool nosetests, you have to add a column between the module and
> the method. Example:
>
>    nosetests tests.unit.test_timeutils:TestIso8601Time.test_west_normalize
>
> Using tox, in most OpenStack projects, adding the name of the failing test
> to the tox command is usually ignored. I guess that it's an issue with
> tox.ini of the project? tox rerun the whole test suite which is usually
> very slow (it takes some minutes even on fast computer). Example:
>
>    tox -e py27
> tests.unit.test_timeutils.TestIso8601Time.test_west_normalize
>
> I try sometimes to activate the virtualenv and then type:
>
>    testr run tests.unit.test_timeutils.TestIso8601Time.test_west_normalize
>
> It usually fails for different reasons.
>
> Example with python-swiftclient. I run unit tests using "tox -e py33. Some
> tests are failing. I enter the virtual environment and type the following
> command to rerun a failing test:
>
>    testr run tests.test_swiftclient.TestPutObject.test_unicode_ok
>
> The test is not run again, no test is run. It's surprising because the
> same command works with Python 2. It's probably a bug in testr?
>
>
>
See the wiki page I gave to you. Some helpful tricks are there. That said,
I never had the issue you mentioned related to only checking one unittest
by providing the path.
When I'm isolating one test, tox -epy27 <pattern> is enough for me.
Run_tests.sh also accepts <pattern>, which is not necessarly the full
python path for the class or the classmethod to be checked.



>
> (3) testscenarios doesn't work with nosetests. It's annoying because for
> the reasons listed above, I prefer to run tests using nosetests. Why do we
> use testscenarios and not something else? Do we plan to support nosetests
> (and other Python test runners) for testscenarios?
>
>
>
You can run testtools without testr. Nosetests has been marked as
non-supported, IIRC.

-Sylvain


> Victor
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140408/352d1eb5/attachment.html>