Open Stack

Hi,

A lot of people would like to see us with more commit gating jobs that
test functionality across the full range of core OpenStack projects.
We've made some progress in that direction, and I think we can start
some limited testing by turning on a gating job for the stable/diablo
branch of several projects.

We have a job on Jenkins that creates a new VM, runs devstack on it, and
then exercise.sh:

https://jenkins.openstack.org/job/dev-gate-integration-tests-devstack-vm/

It will eventually run the tempest test suite once it's ready.  It is
triggered by gerrit changes to the following projects and branches:

openstack/nova              stable/diablo
openstack/glance            stable/diablo
openstack/keyston           stable/diablo
openstack/openstack-ci      master
openstack/python-novaclient master
openstack-dev/devstack      stable/diablo

A change to any of those projects (on those branches) currently triggers
this job, in silent mode, which means it runs on the change before it's
merged into gerrit, but does not vote, and so can not reject the change.
This configuration has been running for a couple of weeks now.

In general, it usually works, but we have seen a few failures, typically
either a failure to deploy a VM from the cloud provider or a part of the
install or setup process that hits the network and encounters a failure.
We'll continue to work on stabilizing it (volunteers welcome!)  In the
mean time, it's fairly easy to retrigger the test in either jenkins or
gerrit.

There are still a number of issues involved in turning this on for
trunk, not only related to stability and determinism, but also to
coordinating simultaneous changes to multiple projects.  However, I
think this is reasonably stable and workable for the stable/diablo
branch.  It will allow us to get some experience with a cross-project
integration test gating job without risking slowing down trunk
development too much.  And it just might catch a bug.

So unless anyone objects, I'd like to disable "silent mode" and make
this an actual gating job for stable/diablo.

-Jim