[openstack-dev] Help us reduce gate resets by being specific about bugs being found

Sean Dague sean at dague.net
Thu Sep 26 14:49:48 UTC 2013


As many folks know, gerrit takes comments of either the form

recheck bug #XXXXX
or
recheck no bug

To kick off the check queue jobs again to handle flakey tests.

The problem is that we're getting a lot more "no bug" than bugs at this 
point. If a failure happens in the OpenStack gate, it's usually an 
actual OpenStack race somewhere. Figuring out what the top races are is 
*really* important to actually fixing those races, as it gives us focus 
on what the top issues are in OpenStack that we need to fix. That makes 
the gate good for everyone, and means less time babysitting your patches 
through merge.

Now that Matt, Joe, and Clark have built the elastic-recheck bot, you 
will often be given a hint in your review about the most probably race 
that it was found. Please confirm the bug looks right before rechecking 
with it, but it should help expedite finding the right issue. 
http://status.openstack.org/rechecks/ is also helpful in seeing what's 
most recently been causing issues.

Here's the score card of how we are doing now at the project level 
(percentage is the percentage of rechecks with a bug, and the fraction 
shows number with a bug / total rechecks issued)

Project Rechecks percentages (last 3000 gerrit reviews)
openstack/requirements                   100% (1 / 1)
openstack/cinder                          78% (25 / 32)
openstack/heat                            66% (8 / 12)
openstack/keystone                        66% (4 / 6)
openstack/python-keystoneclient           66% (4 / 6)
openstack/swift                           52% (10 / 19)
openstack-infra/devstack-gate             50% (2 / 4)
openstack/horizon                         50% (5 / 10)
openstack/python-ceilometerclient         44% (4 / 9)
openstack/python-cinderclient             42% (3 / 7)
openstack/glance                          40% (2 / 5)
openstack-dev/devstack                    38% (7 / 18)
openstack/neutron                         33% (6 / 18)
openstack/python-novaclient               33% (1 / 3)
openstack/nova                            25% (34 / 134)
openstack/tempest                         17% (12 / 69)
openstack/ceilometer                      11% (4 / 34)
stackforge/taskflow                        0% (0 / 1)
openstack/ironic                           0% (0 / 5)
openstack/oslo-incubator                   0% (0 / 3)
openstack-dev/hacking                      0% (0 / 1)
openstack/trove                            0% (0 / 8)
stackforge/savanna                         0% (0 / 1)
openstack-infra/config                     0% (0 / 1)
openstack/python-neutronclient             0% (0 / 1)
stackforge/rally                           0% (0 / 3)


	-Sean

-- 
Sean Dague
http://dague.net



More information about the OpenStack-dev mailing list