Open Stack

On 05/20/2013 06:24 AM, Giulio Fidente wrote:
> Also, not many contributing will expect tags to work like that because
> no test framework (that I know) behaves like that. If you set a tag you
> don't expect another tag to be added silently.
>
> I'd have preferred to keep the tags as a list, setting both, smoke and
> gate, where needed.
>
> But the thing is, I'd like to make good use of this now. Maybe it is
> just the tags naming choice which makes it confusing for me.
>
> As far as I understand:
>   - we use 'gate' for tests to be executed when gating
>   - we do not set any tag for slow running tests
>
> When do we set the smoke tag?

For our purposes:

Tags:

smoke - a hand picked collection of tests that should run in < 10 
minutes on average hardware, and give a good handle on whether or not 
the environment is functioning.

gate - a hand picked collection of tests that should run in < 45 minutes 
in OpenStack CI, and give a broad coverage on features in OpenStack.


Smoke should always be a subset of Gate in our use case.

For our non english friends, maybe Smoke should be turned into 'Quick' 
or something? The smoke tests are definitely just about getting the most 
bang for our buck for short runs.

	-Sean

-- 
Sean Dague
http://dague.net