Open Stack

Steve Baker has started the process of moving Heat tests out of the 
Tempest repository and into the Heat repository, and we're looking for 
some guidance on how they should be packaged in a consistent way. 
Apparently there are a few projects already packaging functional tests 
in the package <projectname>.tests.functional (alongside 
<projectname>.tests.unit for the unit tests).

That strikes me as odd in our context, because while the unit tests run 
against the code in the package in which they are embedded, the 
functional tests run against some entirely different code - whatever 
OpenStack cloud you give it the auth URL and credentials for. So these 
tests run from the outside, just like their ancestors in Tempest do.

There's all kinds of potential confusion here for users and packagers. 
None of it is fatal and all of it can be worked around, but if we 
refrain from doing the thing that makes zero conceptual sense then there 
will be no problem to work around :)

I suspect from reading the previous thread about "In-tree functional 
test vision" that we may actually be dealing with three categories of 
test here rather than two:

* Unit tests that run against the package they are embedded in
* Functional tests that run against the package they are embedded in
* Integration tests that run against a specified cloud

i.e. the tests we are now trying to add to Heat might be qualitatively 
different from the <projectname>.tests.functional suites that already 
exist in a few projects. Perhaps someone from Neutron and/or Swift can 
confirm?

I'd like to propose that tests of the third type get their own top-level 
package with a name of the form <projectname>-integrationtests (second 
choice: <projectname>-tempest on the principle that they're essentially 
plugins for Tempest). How would people feel about standardising that 
across OpenStack?

thanks,
Zane.