Open Stack

On Fri, Aug 30, 2013 at 12:23:03PM -0400, David Kranz wrote:
> Now that we have achieved the goal of parallel tempest in the gate
> using testr we have to be careful that we don't introduce tests that
> are flaky.  This may be obvious to many of you but we should include
> some information in the tempest README. Here is a start.
> Improvements are welcome and I will soon add this to the README.
> 
>  -David
> 
> A new test only has to pass twice to get into the gate. The default
> tenant isolation prevents most tests from trying to access the same
> state in a racy way, but there are some apis, particularly in the
> whitebox and admin area (also watch out for the cli tests) that
> affect more global resources. Such tests can cause race failures. In
> some cases a lock can be used to serialize execution for a set of
> tests. An example is AggregatesAdminTest.  Races between methods in
> a class is not a problem because parallelization is at the class
> level, but if there is a json and xml version of the class there
> could still be a problem. Reviewers need to keep on top of these
> issues.
> 

Thanks for starting this David. This was definitely something that we needed to
add to the tempest docs. Because some of the assumptions that we made before
when things were running serially aren't true in a parallel environment. Just
an FYI I took this description and added a bit more detail to it. I pushed it
out for review here:
https://review.openstack.org/46774

Thanks,

-Matt Treinish