Open Stack

On 11/25/2013 05:19 PM, Matt Riedemann wrote:
> I'll play devil's advocate here and ask this question before someone
> else does.  I'm assuming that the requirement of a 'full' tempest run
> means running this [1].  Is that correct?  It's just confusing sometimes
> because there are other things in Tempest that aren't in the 'full' run,
> like stress tests.
> 
> Assuming that's what 'full' means, it's running API, CLI, third party
> (boto), and scenario tests.  Does it make sense to require a nova virt
> driver's CI to run API tests for keystone, heat and swift?  Or couldn't
> the nova virt driver CI be scoped down to just the compute API tests?
> The argument against that is probably that the network/image/volume
> tests may create instances using nova to do their API testing also.  The
> same would apply for the CLI tests since those are broken down by
> service, i.e. why would I need to run keystone and ceilometer CLI tests
> for a nova virt driver?
> 
> If nothing else, I think we could firm up the wording on the wiki a bit
> around the requirements and what that means for scope.
> 
> [1] https://github.com/openstack/tempest/blob/master/tox.ini#L33
> 

I think the short answer is, "whatever we're running against all Nova
changes in the gate".

I expect that for some drivers, a more specific configuration is going
to be needed to exclude tests for features not implemented in that
driver.  That's fine.

Soon we also need to start solidifying criteria for what features *must*
be implemented in a driver.  I think we've let some drivers in with far
too many features not supported.  That's a separate issue from the CI
requirement, though.

-- 
Russell Bryant