Open Stack

It's clear that lots of projects want 3rd Party CI information on
patches. But it's also clear that 6 months into this experiment with a
lot of 3rd Party CI systems, the Gerrit UI is really not great for this.

A couple of things have fallen out of this. 3rd Party CI bots outnumber
Human comments on changes on some projects (Nova / Neutron). That has an
impact on the readability of the votes section (on a neutron change the
files in the change are rarely above the fold), the readability of the
comments.

3rd Party CI systems haven't become all that reliable. They fall into
the same problems that Jenkins hits with cloud networking, race bugs in
OpenStack, but also new bugs around site configs. It's kind of a
testament to how much we've learned about how to keep the machine
running that the upstream CI system, even with all it's faults, still
trends more reliable than most of the 3rd Party systems.

Commenting in Gerrit was to eventually get towards voting in Gerrit. But
my experience at this point is reviewers are at CI fatigue and are
mostly not paying attention to the votes. Heck, when we're dealing with
a bunch of bugs in the gate most reviewers want to ignore the Jenkins
votes too, which is why you get the recheck grinding behavior.

This has all gone far enough that someone actually wrote a Grease Monkey
script to purge all the 3rd Party CI content out of Jenkins UI. People
are writing mail filters to dump all the notifications. Dan Berange
filters all them out of his gerrit query tools.

It seems what we actually want is a dashboard of these results. We want
them available when we go to Gerrit, but we don't want them in Gerrit
itself.

What if 3rd Party CI didn't vote in Gerrit? What if it instead published
to some 3rd party test reporting site (a thing that doesn't yet exist).
Gerrit has the facility so that we could inject the dashboard content
for this in Gerrit in a little table somewhere, but the data would
fundamentally live outside of Gerrit. It would also mean that all the
aggregate reporting of 3rd Party CI that's being done in custom gerrit
scripts, could be integrated directly into such a thing.

I'm not signing up for this particular mission, but I wanted to stick it
out there to figure out if the idea had merrit, and if it did, if it
excited anyone to enough to dive on it.

	-Sean

-- 
Sean Dague
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140627/043342bc/attachment.pgp>