Open Stack

Now that we have achieved the goal of parallel tempest in the gate using 
testr we have to be careful that we don't introduce tests that are 
flaky.  This may be obvious to many of you but we should include some 
information in the tempest README. Here is a start. Improvements are 
welcome and I will soon add this to the README.

  -David

A new test only has to pass twice to get into the gate. The default 
tenant isolation prevents most tests from trying to access the same 
state in a racy way, but there are some apis, particularly in the 
whitebox and admin area (also watch out for the cli tests) that affect 
more global resources. Such tests can cause race failures. In some cases 
a lock can be used to serialize execution for a set of tests. An example 
is AggregatesAdminTest.  Races between methods in a class is not a 
problem because parallelization is at the class level, but if there is a 
json and xml version of the class there could still be a problem. 
Reviewers need to keep on top of these issues.