[openstack-qa] some stats for existing tempest tests

Jay Pipes jaypipes at gmail.com
Thu May 16 19:44:05 UTC 2013


No, we don't need all the below tags, and yes we need cleanup :)
Comments inline on specific suggestions...

On 05/16/2013 03:24 PM, Giampaolo Lauria wrote:
> Hi,
> 
> while working on
> https://blueprints.launchpad.net/tempest/+spec/set-gate-attribute, I
> decided to write a script to get a picture of what we currently have for
> tags and attributes in our tests. Here is what I have found:
> 
> Total tests found:  583
> 
> Tag                                   Total # of instances
> --------------------                  --------------------
> 'NO TAG'                     226
> EC2                                   6
> S3                                    4
> gate                                  20
> live-migration                1

Remove this.

> lp1170718                     1
> lp912922                              2

Remove these.

> nagative                              1
> negative                              168
> positive                              110

I've never liked either the negative or positive categorization of
tests. I don't see the point.

> regression                    1

Bad tag. Regression for what? Regressions are fixed over time, so this
tag is a perfect example of something that just adds technical debt to
the project.

> server-addresses              4

Remove this.

> slow                                  1

Slow when? Slow now? Slow at some point in the past? Always be slow?
This is just technical debt.

> smoke                                 82

WAY too many things are tagged with smoke. All smoke tags should be
removed, IMO, and the scenario tests should be considered the
"acceptance" or "promotion" or "smoke" tests.

> whitebox                              2
> 
> Attribute                             Total # of instances
> --------------------                  --------------------
> bug                                   3
> category                              5
> type                                  370

It would be great IMO if we could have separate decorators for @bug,
@gate, and @type and limited the tags accordingly.

Best,
-jay

> Do we need all the above tags and attributes or should we do some cleanup?
> 
> Thanks,
> Giampaolo Lauria
> Software Engineer
> IBM Linux Technology Center
> 
> 
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
> 



More information about the openstack-qa mailing list