Open Stack

On 25/07/14 11:18, Sean Dague wrote:
> On 07/25/2014 10:01 AM, Steven Hardy wrote:
>> On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
>> <snip>
>>>   * Put the burden for a bunch of these tests back on the projects as
>>>     "functional" tests. Basically a custom devstack environment that a
>>>     project can create with a set of services that they minimally need
>>>     to do their job. These functional tests will live in the project
>>>     tree, not in Tempest, so can be atomically landed as part of the
>>>     project normal development process.
>> +1 - FWIW I don't think the current process where we require tempest
>> cores to review our project test cases is working well, so allowing
>> projects to own their own tests will be a major improvement.
>>
>> In terms of how this works in practice, will the in-tree tests still be run
>> via tempest, e.g will there be a (relatively) stable tempest api we can
>> develop the tests against, as Angus has already mentioned?
> No, not run by tempest, not using tempest code.
>
> The vision is that you'd have:
>
> heat/tests/functional/....
>
> And tox -e functional would run them. It would require some config for
> end points. But the point being it would be fully owned by the project
> team. That it could do both blackbox/whitebox testing (and because it's
> in the project tree would know things like the data model and could poke
> behind the scenes).
>
> The tight coupling of everything is part of what's gotten us into these
> deadlocks, decoupling here is really required in order to reduce the
> fragility of the system.
>
>
Since the tempest scenario orchestration tests use heatclient then
hopefully it wouldn't be too much effort to forklift them into
heat/tests/functional without any tempest dependencies.

We can leave the orchestration api tests where they are until the
tempest-lib process results in something ready to use.