Open Stack

On 06/09/2013 05:49 PM, Daryl Walleck wrote:
> The only other time I can think of when having an isolated tenant would be a must would be when testing usage of quotas, as it'd be impossible to keep a tenant's quota usage in one state with multiple concurrent tests running. Networks and Cinder would have similar concerns, as would any integration testing with Ceilometer.

I'm not sure I'm getting my point across about why isolated tenants are 
absolutely essential for parallel execution of Tempest...

If you have test cases running within the same tenant that are testing 
the creation, modification and destruction of resources (say, server 
instances), and test cases that are testing the listing and showing of 
resources, and you run them in parallel, then the tests need to take 
into account what the other test cases are doing. This means more 
complex code that must be injected into the listing test cases to filter 
out unexpected results that suddenly appear in the results for things 
like GET /servers because other test cases have added or destroyed 
resources between the time when the listing test case originally 
determined the number of server resources before adding the server 
resources that its own test case uses when doing assertions.

The reason tenant isolation was added to Tempest was to get rid of this 
issue.

I honestly feel like we are debating getting rid of tenant isolation for 
silly micro-optimization reasons and completely missing the big picture 
that isolation/containerization of the test case fixtures means they can 
be run in parallel.

Best,
-jay