[openstack-dev] [qa] Plan for failing "successful" tempest jobs when new ERRORs appear in logs

David Kranz dkranz at redhat.com
Mon Nov 18 22:58:47 UTC 2013


So we are close to being able to start doing this. The current whitelist 
is here 
https://github.com/openstack/tempest/blob/master/etc/whitelist.yaml. I 
have a "find-errors" script that watches for successful builds and pulls 
out the non-whitelisted errors. For the past few weeks I have been doing 
the following:

1. Run find-errors
2. File bugs on any new errors
3. Add to whitelist
4. Repeat

There are still some very flaky cases. I will do one more iteration of 
this. Right now this script 
https://github.com/openstack/tempest/blob/master/tools/check_logs.py 
dumps non-whitelisted errors to the console log but
always returns success. The question now is how long should all jobs run 
with no new errors showing, before changing check_logs.py to fail if 
there are any new errors?

  -David



More information about the OpenStack-dev mailing list