Open Stack

On 07/25/2014 10:01 AM, Steven Hardy wrote:
> On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
> <snip>
>>   * Put the burden for a bunch of these tests back on the projects as
>>     "functional" tests. Basically a custom devstack environment that a
>>     project can create with a set of services that they minimally need
>>     to do their job. These functional tests will live in the project
>>     tree, not in Tempest, so can be atomically landed as part of the
>>     project normal development process.
> 
> +1 - FWIW I don't think the current process where we require tempest
> cores to review our project test cases is working well, so allowing
> projects to own their own tests will be a major improvement.
> 
> In terms of how this works in practice, will the in-tree tests still be run
> via tempest, e.g will there be a (relatively) stable tempest api we can
> develop the tests against, as Angus has already mentioned?

No, not run by tempest, not using tempest code.

The vision is that you'd have:

heat/tests/functional/....

And tox -e functional would run them. It would require some config for
end points. But the point being it would be fully owned by the project
team. That it could do both blackbox/whitebox testing (and because it's
in the project tree would know things like the data model and could poke
behind the scenes).

The tight coupling of everything is part of what's gotten us into these
deadlocks, decoupling here is really required in order to reduce the
fragility of the system.

	-Sean

-- 
Sean Dague
http://dague.net