Open Stack

On 7 January 2014 07:41, David Kranz <dkranz at redhat.com> wrote:
> Thanks to all who looked at https://review.openstack.org/#/c/64733/. There
> were a few minor issues I will address but the biggest one was the
> suggestion to run each variation as a separate test case using
> testscenarios. After looking into that I see a problem with this use case.
> Many of these negative tests need to allocate resources such as servers that
> are then referenced in the test cases. Our tests currently do stuff like
> that in setUpClass. But testscenarios, unless I misunderstand the code,
> replicates the whole class which means the resource setup would also be
> duplicated for each test. I think we need to share allocated resources
> across the potentially large number of negative variations, and see two
> options for that:

You don't understand the code :). testscenarios doesn't affect the
class of tests. As long as the resulting suite is wrapped in a
setUpClass compatible TestSuite object, setUpClass should keep
working.

> 1. Some magic similar to testscenarios but that creates test methods with
> varying parameters instead of test classes with varying attributes.
>     Leave allocation in setUpClass.

-1 No different to testscenarios, just different to be different.

> 2. Using testscenarios as-is but move the allocation/cleanup to the module
> level (fixtures?)
>     This would be a little complicated because a bunch of the setup stuff is
> defined as methods on the test base classes. Also, the classes would
>     run in parallel.

Long term perhaps, not needed for this work.

-Rob