Open Stack

On 07/04/2013 04:59 PM, Monty Taylor wrote:
> On 07/04/2013 03:19 PM, David Kranz wrote:
>> While approving https://review.openstack.org/#/c/35394/, Chris Yeoh
>> asked if there was any intent for the stress tests to clean up after
>> themselves. My intention was "no". The philosophy is that the stress
>> tests run for some period of time. If there are any errors in the api
>> calls being used, the test fails. Periodically, the log files are
>> scanned for ERRORs. If any are found, the test fails. It the test times
>> out with no api errors or log errors then the test passes. IMO, adding
>> cleanup just makes writing the test cases more complicated without any
>> real benefit. There is a script in the stress directory that tries to
>> clean up all existing resources but it is not really possible in general
>> now because not all resource-related apis have the "all_tenants" option.
>>
>> Due to existing bugs in some projects, ERRORs appear in the logs during
>> Tempest runs that are successful. My thought was to start running the
>> stress tests periodically and file bugs as necessary. It may be
>> necessary to make the log scanner skip certain error messages until the
>> bugs are fixed so we can continue to run the tests and find more bugs.
>
> I would love to see successful runs cease to have ERRORs or tracebacks
> in the logs. Similar to getting code warning-free on compile, having the
> logs only have errors in them if there's an actual operational error (as
> opposed to a person requesting an invalid user) would greatly improve
> many things.

As you know, I share your passion for warning-free compilation in 
C/C++... the two of us spent months getting Drizzle's server compile to 
be warning free :)

++ to fixing places that write ERRORs when there is no real error that 
has occurred -- like a 404 returned for the *primary* requested resource.

-jay