Open Stack

On 08/05/2013 07:40 AM, Koderer, Marc wrote:
> Hi all,
>
> After some refactoring work in tempest/stress I would like to raise a general
> question since I have the feeling we have different opinions about the purpose
> of tempest stress test.
> Giulio already put this topic on the agenda for the next QA meeting and I just
> want that we use the time in between to think about the problem ;)
>
> Please have a look to the discussions in:
>    https://review.openstack.org/#/c/39752/
>    https://review.openstack.org/#/c/38980/
>
> IMHO a stress test is not a independent test area (like api-tests, scenario
> test etc) it's just a way how tests are processed. So in theory any small
> API-test could be used as a stress test and any scenario test could be used
> too.
>
> I see two use cases for stress tests:
>
>    - As a developer I want to find bugs that occur under load (like raise conditions)
>      --> Leads to many small and concurrent api tests
>    - As OPS/QA I want to generate load that simulates real life load in a
>      production-like system
>      --> Leads to concurrent scenario test
>
> Kind Regards
> Marc
>
IMO the most important use case for stress tests is the first. The idea 
is to make something happen, like a race condition, that would only 
happen rarely, and be much harder to debug, under real life load. 
Simulating real life load is important for performance tuning and the 
like but I think is a bit different. I don't see why the stress 
framework should not be able to support concurrent scenario tests though.

  -David