Open Stack

On 3/10/2017 3:02 PM, Andrea Frittoli wrote:
>
> We had a couple of sessions related to this topic at the PTG [0][1].
>
> We agreed that we want to still maintain integration tests only in
> Tempest, which means that API micro versions that have no integration
> impact can be tested via functional tests.

To be clear, "integration" here means tests that span operations across 
multiple services, correct? Like a compute test that first creates a 
port in the networking service and a volume in the block storage service 
and then uses those to create a server and maybe take a snapshot of it 
which is then verified was uploaded to the image service.

The non-integration things are self-contained in a single service, like 
if all you need to do is create an aggregate, show it's details and 
validate the response, at a particular microversion, we can just do that 
in nova functional tests, and it's not necessary in Tempest.

It might be worth having a definition of this policy in the Tempest docs 
so when people ask this question again you can just point at the docs.

>
> In terms of which versions we test in the gate, for nova we always run
> with min_microversion = None and max_microversion = latest, which means
> that all tests will be executed.
> Since micro versions are incremental, and each micro version usually
> involves no or one test on Tempest side, I think it will be a while
> before this becomes an issue for the common gate.

We test max_microversion=latest only on master. On the devstack stable 
branch we cap the max_microversion, e.g.:

https://github.com/openstack-dev/devstack/blob/stable/newton/lib/tempest#L339

--

Thanks,

Matt