Open Stack

Hi,

Unfortunately, it looks like each regressions introduced by releases of 
Oslo libraries are still common :-( We already have tools to detect 
regressions, but they are run manually. It would be nice to automate 
these tests and run them more frequently (at least one per week, or even 
daily?).

There are tools to run unit tests of projects like neutronclient or nova 
on the development version of oslo.* libraries. They take up to 12 hours 
to run all tests. Example of tools:

- "tox -e venv -- oslo_run_pre_release_tests" in each Oslo project
- tools/oslo_run_cross_tests in oslotest

To prepare the latest bunch of releases, dims wrote two patches to run 
nova unit tests and tempest:

* https://review.openstack.org/#/c/186413/
* https://review.openstack.org/#/c/186418/

Unfortunately, the stable version of some oslo.* projects were used 
instead of the development version, and 2 regressions were missed :-/

It would nice to automate a job running cinder, nova and neutron unit 
tests and tempest on the development versions (git master branch) of all 
oslo.* projects. We can start with something simple: run 
tools/oslo_run_cross_tests and send the result by email every day to 
some people interested by the result (ex: Oslo liaisons, or just me if 
nobody cares). It would be nice to have a dedicated resource in the 
OpenStack infra for such job.

I proposed to run "all" tests using Gerrit for each patch send to an 
Oslo project, but it would use too much resources of the OpenStack infra.

Another idea would be to write a check job dedicated to Oslo releases 
and use Gerrit to prepare a release (at least to tag a version in Git).

Victor