Open Stack

On Fri, May 23, 2014 at 09:34:35AM +0900, GHANSHYAM MANN wrote:
> Tempest API’s XML test will run with 1 test failure when run with Nosetest.
> When we run the tempest with nosetest command (ex -nosetest –v
> ./tempest/api/compute/test_xyz.py), it run *BaseComputeTest
> ::create_test_server & create_test_server_group *functions also as test
> with any of the API tests. Because nose looks for tests in directories and
> modules whose names start with *test* and *Test*, or contain a '_', '.',
> or '-' followed by test or Test.

So this is part of the many reasons we don't support or advise running tempest
with nosetests. I brought this up a few months ago when I tried to be a bit
more aggressive about this and block nose from being run at all. [1] Is there
any particular reason you are still trying to run Tempest with nosetests?

> 
> 
> 
> So for each API XML test, failure will happen in  create_test_server_group
> function as  it’s XML client is not implemented (as expected).
> 
> 
> 
> To fix/stop these kind of issue, one solution is to have some policy for
> Tempest not to name non test function with test_, Test_ etc so that nose
> would not consider those as test case.

So trying to accommodate nose especially on this isn't going to happen. The
python docs clearly state test_* named methods are what should be used so that
the test driver recognizes them as tests. Nose is what is being overzealous
and causing issues; the bug is in nose not tempest. I think this suggested patch
or something similar has been pushed at least twice already. The problem is even
after making this change large chunks of tempest still won't work because of
other things with nose. (like testscenarios) It's really not worth the review
effort to merge a change like this. We've been actually moving in the other
direction too and removed all the in tree references and workarounds related to
nose during Icehouse.

For a project of tempest's size and growth rate trying to workaround all the
quirks of every test runner isn't really feasible especially because we're not
running with anything but testr in the gate. That being said I think nose might
be a slightly special case in that I know it doesn't work for several reasons.
(only because it used to be the default) Other test runners probably wouldn't
have nearly as many issues, if any at all.

> 
> 
> 
> Looking forward to have more opinions on this.
> 
> 


-Matt Treinish

[1] http://lists.openstack.org/pipermail/openstack-dev/2014-January/024673.html