Open Stack

On 09/13/2013 06:20 AM, Sean Dague wrote:
> Tempest has had a set of tests called whitebox that came in late in the
> Folsom cycle. The idea being that you gave tempest db creds, and these
> tests would check for the db being in a certain state after certain API
> calls. It would also inject certain database state to get specific API
> results. This was helpful in nailing some state races at the time.
> 
> But since then these have largely bitrotted, especially with the db
> schemas moving as fast as they do. As we started trying to make sure
> everything in the Tempest tree was actually running regularly these were
> failing a lot. David Kranz and I recently looked at them, and what we
> had seemed to be largely Nova state change tests (doesn't affect any
> other service).
> 
> Going parallel made this worse, because isolation against the db is an
> entirely different kind of problem than tenant isolation.
> 
> So, that being said the call we made on IRC and in the QA meeting is to
> get rid of them, and instead port the checks that we had into Nova unit
> tests (which currently covered about 50% of the state checks that we in
> whitebox). This brings it closer to the project which seems the right
> approach.
> 
> We are tracking this with - https://bugs.launchpad.net/tempest/+bug/1222852
> 
> The Tempest half is now done, the Nova half has a volunteer.
> 
> While we are pretty sure that we got everyone's take on this, now is the
> time to raise final objections to the approach.

I think this is an excellent plan.