Open Stack

----- Original Message -----
> On Wed, 2017-05-03 at 17:53 -0400, Emilien Macchi wrote:
> > (cross-posting)

> 
> > Instead of running the Pingtest, we would execute a Tempest Scenario
> > that boot an instance from volume (like Pingstest is already doing)
> > and see how it goes (in term of coverage and runtime).
> > I volunteer to kick-off the work with someone more expert than I am
> > with quickstart (Arx maybe?).
> > 
> > Another iteration could be to start building an easy interface to
> > select which Tempest tests we want a TripleO CI job to run and plug
> > it
> > to our CI tooling (tripleo-quickstart I presume).
> 
> Running a subset of Tempest tests isn't the same thing as designing
> (and owning) your own test suite that targets the things that mean the
> most to our community (namely speed and coverage). Even giving up 5-10
> minutes of runtime...just to be able to run Tempest isn't something
> that some of us would be willing to do.

As I mentioned, you can do it with Tempest (the library). You can have your own test suite that does exactly what you are asking (namely, a set of scenario tests based on Heat which targets the TripleO use case) in a Tempest plugin and there is no absolute reason that those tests should add 5-10 minutes of runtime compared to pingtest. 

It/they would be exactly pingtest, only implemented using a different library and running with a different runner, with the *exact* same run time. 

Obvious advantages: only one technology used to run tests, so if anyone else want to run additional tests, there is no need to maintain two code paths; reuse on a big and proven library of test and test runner tools.

Ciao
-- 
Luigi