[openstack-qa] some stats for existing tempest tests

David Kranz david.kranz at qrclab.com
Thu May 16 20:57:43 UTC 2013


On 5/16/2013 4:30 PM, Sean Dague wrote:
> SNIP...

>>> smoke                                 82
>>
>> WAY too many things are tagged with smoke. All smoke tags should be
>> removed, IMO, and the scenario tests should be considered the
>> "acceptance" or "promotion" or "smoke" tests.
>
> So that's kind of a good point. Once we get the restructure in, one of 
> our goals is to ensure scenario >= devstack exercise scripts in value.
>
> Once that is done, I think we can drop smoke.
>
> Right now, smoke is still actively used in the gate for Quantum, as 
> quantum can't pass full gate.
>
>     -Sean
>
The problem with this is that 'smoke' was supposed to be fast. The 
scenario tests are part of the gate and I'm not sure we want to put such
a time budget on them. But we could limit a smoke tag to a subset of 
scenario tests, perhaps tailoring some of them for this purpose: maximum 
coverage for minimal time.

  -David



More information about the openstack-qa mailing list