Open Stack

While approving https://review.openstack.org/#/c/35394/, Chris Yeoh asked if there was any intent for the stress tests to clean up after themselves. My intention was "no". The philosophy is that the stress tests run for some period of time. If there are any errors in the api calls being used, the test fails. Periodically, the log files are scanned for ERRORs. If any are found, the test fails. It the test times out with no api errors or log errors then the test passes. IMO, adding cleanup just makes writing the test cases more complicated without any real benefit. There is a script in the stress directory that tries to clean up all existing resources but it is not really possible in general now because not all resource-related apis have the "all_tenants" option.

Due to existing bugs in some projects, ERRORs appear in the logs during Tempest runs that are successful. My thought was to start running the stress tests periodically and file bugs as necessary. It may be necessary to make the log scanner skip certain error messages until the bugs are fixed so we can continue to run the tests and find more bugs.

Please chime in if you have other ideas.

  -David