Open Stack

On Monday, November 25, 2013 4:37:29 PM, Russell Bryant wrote:
> On 11/25/2013 05:19 PM, Matt Riedemann wrote:
>> I'll play devil's advocate here and ask this question before someone
>> else does.  I'm assuming that the requirement of a 'full' tempest run
>> means running this [1].  Is that correct?  It's just confusing sometimes
>> because there are other things in Tempest that aren't in the 'full' run,
>> like stress tests.
>>
>> Assuming that's what 'full' means, it's running API, CLI, third party
>> (boto), and scenario tests.  Does it make sense to require a nova virt
>> driver's CI to run API tests for keystone, heat and swift?  Or couldn't
>> the nova virt driver CI be scoped down to just the compute API tests?
>> The argument against that is probably that the network/image/volume
>> tests may create instances using nova to do their API testing also.  The
>> same would apply for the CLI tests since those are broken down by
>> service, i.e. why would I need to run keystone and ceilometer CLI tests
>> for a nova virt driver?
>>
>> If nothing else, I think we could firm up the wording on the wiki a bit
>> around the requirements and what that means for scope.
>>
>> [1] https://github.com/openstack/tempest/blob/master/tox.ini#L33
>>
>
> I think the short answer is, "whatever we're running against all Nova
> changes in the gate".

Maybe a silly question, but is what is run against the check queue any 
different from the gate queue?

>
> I expect that for some drivers, a more specific configuration is going
> to be needed to exclude tests for features not implemented in that
> driver.  That's fine.
>
> Soon we also need to start solidifying criteria for what features *must*
> be implemented in a driver.  I think we've let some drivers in with far
> too many features not supported.  That's a separate issue from the CI
> requirement, though.
>

--

Thanks,

Matt Riedemann