Open Stack

On 07/04/2013 03:19 PM, David Kranz wrote:
> While approving https://review.openstack.org/#/c/35394/, Chris Yeoh
> asked if there was any intent for the stress tests to clean up after
> themselves. My intention was "no". The philosophy is that the stress
> tests run for some period of time. If there are any errors in the api
> calls being used, the test fails. Periodically, the log files are
> scanned for ERRORs. If any are found, the test fails. It the test times
> out with no api errors or log errors then the test passes. IMO, adding
> cleanup just makes writing the test cases more complicated without any
> real benefit. There is a script in the stress directory that tries to
> clean up all existing resources but it is not really possible in general
> now because not all resource-related apis have the "all_tenants" option.
> 
> Due to existing bugs in some projects, ERRORs appear in the logs during
> Tempest runs that are successful. My thought was to start running the
> stress tests periodically and file bugs as necessary. It may be
> necessary to make the log scanner skip certain error messages until the
> bugs are fixed so we can continue to run the tests and find more bugs.

I would love to see successful runs cease to have ERRORs or tracebacks
in the logs. Similar to getting code warning-free on compile, having the
logs only have errors in them if there's an actual operational error (as
opposed to a person requesting an invalid user) would greatly improve
many things.