Open Stack

Sorry for cross-posting to both mailing lists, but there's lots of folks
working on setting up third-party testing platforms that are not members
of the openstack-infra ML...

tl;dr
-----

The third party testing documentation [1] has requirements [2] that
include the ability to trigger a recheck based on a gerrit comment.

Unfortunately, the Gerrit Jenkins Trigger plugin [3] does not have the
ability to trigger job runs based on a regex-filtered comment (only on
the existence of any new comment to the code review).

Therefore, we either should:

a) Relax the requirement that the third party system trigger test
re-runs when a comment including the word "recheck" appears in the
Gerrit event stream

b) Modify the Jenkins Gerrit plugin to support regex filtering on the
comment text (in the same way that it currently supports regex filtering
on the project name)

or

c) Add documentation to the third party testing pages that explains how
to use Zuul as a replacement for the Jenkins Gerrit plugin.

I propose we do a) for the short term, and I'll work on c) long term.
However, I'm throwing this out there just in case there are some Java
and Jenkins whizzes out there that could get b) done in a jiffy.

details
-------

OK, so I've been putting together documentation on how to set up an
external Jenkins platform that is "linked" [4] with the upstream
OpenStack CI system.

Recently, I wrote an article detailing how the upstream CI system
worked, including a lot of the gory details from the
openstack-infra/config project's files. [5]

I've been working on a follow-up article that goes through how to set up
a Jenkins system, and in writing that article, I created a source
repository [6] that contains scripts, instructions and Puppet modules
that set up a Jenkins system, the Jenkins Job Builder tool, and
installs/configures the Jenkins Gerrit plugin [7].

I planned to use the Jenkins Gerrit plugin as the mechanism that
triggers Jenkins jobs on the external system based on gerrit events
published by the OpenStack review.openstack.org Gerrit service. In
addition to being mentioned in the third party documentation, Jenkins
Job Builder has the ability to construct Jenkins jobs that are triggered
by the Jenkins Gerrit plugin [8].

Unforunately, I've run into a bit of a snag.

The third party testing documentation has requirements that include the
ability to trigger a recheck based on a gerrit comment:

<quote>
Support recheck to request re-running a test.
 * Support the following syntaxes recheck no bug and recheck bug ###.
 * Recheck means recheck everything. A single recheck comment should
re-trigger all testing systems.
</quote>

The documentation has a section on using the Gerrit Jenkins Trigger
plugin [3] to accept notifications from the upstream OpenStack Gerrit
instance.

But unfortunately, the Jenkins Gerrit plugin does not support the
ability to trigger a re-run of a job given a regex match of the word
"recheck". :(

So, we either need to a) change the requirements of third party testers,
b) enhance the Jenkins Gerrit plugin with the missing functionality, or
c) add documentation on how to set up Zuul as the triggering system
instead of the Jenkins Gerrit plugin.

I'm happy to work on c), but I think relaxing the restriction (a) is
probably needed short-term.

Best,
-jay

[1] http://ci.openstack.org/third_party.html
[2] http://ci.openstack.org/third_party.html#requirements
[3]
http://ci.openstack.org/third_party.html#the-jenkins-gerrit-trigger-plugin-way
[4] By "linked" I mean it both reads from the OpenStack Gerrit system
and writes (adds comments) to it
[5] http://www.joinfu.com/2014/01/understanding-the-openstack-ci-system/
[6] http://github.com/jaypipes/os-ext-testing
[7] https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger
[8]
https://github.com/openstack-infra/jenkins-job-builder/blob/master/jenkins_jobs/modules/triggers.py#L121