Open Stack

Hello folks,

I'm working on the automated ui testing blueprint[1], and I think we
need to change the way we ship our tempest tests.

Here is where things stand at the moment:

* We have a kolla image for tempest
* This image contains the tempest rpm, and the openstack-tempest-all rpm
* The openstack-tempest-all package in turn contains all of the
  openstack tempest plugins
* Each of the plugins is shipped as an rpm

So, in order for a new test in tempest-tripleo-ui to appear in CI we
have to go through at least the following tests:

* New tempest-tripleo-ui rpm
* New openstack-tempest-all rpm
* New tempest kolla image

This could easily take a week, if not more.

What I would like to build is something like the following:

* Add an option to the tempest-setup.sh script in tripleo-quickstart to
  refresh all tempest plugins from git before running any tests
* Optionally specify a zuul change for any of the plugins being
  refreshed
* Hook up the test job to patches in tripleo-ui (which tests in
  tempest-tripleo-ui are testing) so that I can run a fix and its test
  in a single CI job

This would allow the tripleo-ui team to develop code and tests at the
same time, and prevent breakage before a patch is even merged.

Here are a few questions:

* Do you think this is a good idea?
* Could we accomplish this by some other, simple mechanism?

Any helpful suggestions, corrections, and feedback are much appreciated.

Thanks

Honza Pokorny

[1]: https://blueprints.launchpad.net/tripleo/+spec/automated-ui-testing