[openstack-dev] [nova][testing] How to run a subset of py34 unit tests
mtreinish at kortar.org
Sat Jan 23 03:14:07 UTC 2016
On Fri, Jan 22, 2016 at 06:46:48PM -0800, melanie witt wrote:
> Hi everyone,
> I noticed because of the way we run the py34 tests in tox.ini, I'm not able to specify a filter regex the way I normally do as a positional arg, for example: 'tox -epy34 nova.tests.unit.network' doesn't filter and it runs everything. ('tox -epy27 nova.tests.unit.network' will only run tests that match nova.tests.unit.network).
> I couldn't figure out how we could add something to tox.ini to make it work -- we're calling ostestr with the --blacklist_file option. I'm not completely clear on what 'ostestr --blacklist_file <file> --regex <regex>' does but I couldn't get it to do what I want. From the documentation , it adds --regex to the regex created from the --blacklist_file. The regex from the blacklist file looks something like this '^((?!blacklistedstuff).)*$' and if I can only append to it, the best I could do was a positive lookbehind but that can't match at the beginning of a line. (For example, I tried "ostestr --blacklist_file tests-py3.txt --regex '(?<=network)'" and it matched all the non-blacklisted tests that ended with the word "network"). It seems like what I would need is for --regex to do another re.search() and match the line only if the previous regex from the blacklist also matched.
So os-testr literally just generates a regex  and passes that to testr
directly when it's called with subprocess.  os-testr still relies on testr
to do the actual test selection.
This does sound like a real bug in os-testr's regex generation, can you file
a bug here:
Although, now that I look at the list it definitely looks like this one,
which I had forgotten about:
It sounds like that bug proposes a direction so if you could patch os-testr
to give that a try and see if that fixes your issue. If it does then we can
just land that in os-testr.
However, I'm wondering if it makes more sense to change how the selection
actually works with os-testr. This regex generation approach does get pretty
unwieldy and seems very error prone. I'm thinking it might make more sense to
generate a test-list and then have os-testr filter that list itself and then
pass the pruned list to --load-list in testr run.
>  http://docs.openstack.org/developer/os-testr/ostestr.html#test-selection
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 819 bytes
Desc: not available
More information about the OpenStack-dev