Open Stack

+1 to keep a short running set of tests (e.g. smoke) 

There was as well a proposal to replace devstack exercises with a tempest
smoke test.

Andrea

-----Original Message-----
From: Sean Dague [mailto:sean at dague.net] 
Sent: 15 May 2013 18:18
To: openstack-qa at lists.openstack.org
Subject: Re: [openstack-qa] tempest run length - need a gate tag - call for
help

On 05/15/2013 11:12 AM, David Kranz wrote:
> On 5/11/2013 10:57 PM, Sean Dague wrote:
>> Any assistance would be good.
>>
>> Right now we really just need 'gate' attr added to basically all the 
>> non skipped methods, we can prune later. Once 'gate' looks to be ~ 
>> full, we can flip over check and gate to use that.
>>
>> I think long term the approach we're going to need to go with is 3 
>> sets of tests:
>>
>> smoke (< 10 mins)
>> gate (< 45 mins)
>> full (everything)
>>
>> All projects gate on gate
>>
>> Periodic runs of full - daily, more often?
>>
>> Tempest check runs full (but not gate), it's advisory.
>>
>> Some on demand facility for people to run full.
>>
>> At this point I'm not adding my +2 to any more tests (only approving 
>> fixes to existing tests) until we get gate tag in, as I don't think 
>> we should be running any longer than we currently are.
>>
>>     -Sean
>>
> This message talks about "smoke" but I am not clear what it is 
> actually used for. I don't see a reason to run just smoke tests as 
> part of some automated process.
> So, who will run just smoke tests and why will they be run? I usually 
> think of a smoke test as something I can run before committing or 
> checkpointing my work as a sanity check but to be useful for that it 
> would want to be shorter than 10 min I think. I wonder if we really 
> need a tempest smoke tag at this point.

In reality, it's still needed for quantum gate, it doesn't pass the full
gate now. I also don't think we can take the full hit of tempest on grenade
tests, so smoke should be used for that.

I agree, smoke should be < 10 minutes. We're close right now, probably need
to do a little trimming.

	-Sean

--
Sean Dague
http://dague.net

_______________________________________________
openstack-qa mailing list
openstack-qa at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6202 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-qa/attachments/20130515/1ec5fec8/attachment.bin>