[openstack-dev] [qa] Plan for failing "successful" tempest jobs when new ERRORs appear in logs

Joe Gordon joe.gordon0 at gmail.com
Mon Nov 25 19:08:05 UTC 2013


On Mon, Nov 25, 2013 at 11:06 AM, Joe Gordon <joe.gordon0 at gmail.com> wrote:

>
>
>
> On Mon, Nov 18, 2013 at 2:58 PM, David Kranz <dkranz at redhat.com> wrote:
>
>> So we are close to being able to start doing this. The current whitelist
>> is here https://github.com/openstack/tempest/blob/master/etc/
>> whitelist.yaml. I have a "find-errors" script that watches for
>> successful builds and pulls out the non-whitelisted errors. For the past
>> few weeks I have been doing the following:
>>
>> 1. Run find-errors
>> 2. File bugs on any new errors
>> 3. Add to whitelist
>> 4. Repeat
>>
>> There are still some very flaky cases. I will do one more iteration of
>> this. Right now this script https://github.com/openstack/
>> tempest/blob/master/tools/check_logs.py dumps non-whitelisted errors to
>> the console log but
>> always returns success. The question now is how long should all jobs run
>> with no new errors showing, before changing check_logs.py to fail if there
>> are any new errors?
>>
>
> The sooner the better.
>
>


Also this is awesome. I expect this to significantly help with keeping the
gate stable.



>
>>  -David
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131125/6ababb7c/attachment.html>


More information about the OpenStack-dev mailing list