Open Stack

The ec2 and s3 attributes not (just) for annotating the test type. (they are even not type='foo', they could be services=['ec2', 's3'])

They are used in the skip logic. The test selecting ability by those just a secondary capability.

I wanted to extend this in the future, I mean we should skip test based on services not expected to present in the test environment.
Since tempest supposed to be integration test tool, test cases can have multiple service dependencies (glance<-- compute <--ssh--network).

Additional Note:
We should remove completely the automated service detection, incorrect code changes leads to false positive test suite pass.

I really would like to have the capability, to run a function before all test cases and another function _after_ all test cases.
I would prefer not to miss use some existing solution or miss using a design pattern as an anti pattern in order to achieve this.

Regards,
Attila



----- Original Message -----
From: "Christopher Yeoh" <cyeoh at au1.ibm.com>
To: "Sean Dague" <sdague at linux.vnet.ibm.com>
Cc: openstack-qa at lists.openstack.org
Sent: Friday, February 8, 2013 6:58:51 AM
Subject: Re: [openstack-qa] tempest attribute replacement

On Thu, 07 Feb 2013 14:27:14 -0500
Sean Dague <sdague at linux.vnet.ibm.com> wrote:

> There are some review comments in 
> https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/speed-up-tempest,n,z 
> and we talked about it at the qa meeting today.
> 
> I think Attila brings up a good point that we can't break existing
> nose functionality with this until the alternative is available.
> 
> So I'm going to suggest the following.
> 
> 1) lets build own own @tempest.attr decorator that will set both nose 
> and testr attrs.

I have submitted a changeset which implements this:

https://review.openstack.org/#/c/21492/

It works, but I'm far from sure its the best way to implement it
so any feedback is very welcome.

> 2) lets not deleted the positive / negative attrs yet, it's meta data 
> that I find a little useful in looking at the tests (even if it's not 
> complete).

The changeset above will allow for all the various attr's to remain as
although it always applies the nose attr decorator, it only applies the
testtools one if its the smoke one. I'm not convinced that its a good
idea to apply positive/negative/whitebox ones since they are
inconsistently applied.

As an aside I think we should make a decision as to whether
positive/negative/ec2/whitebox attributes are worth having. I think we
should either always have them (and a regression test for this would
be great) or not have them at all as otherwise it becomes
rather confusing for people who don't know the history - eg "this test
is meant to fail, but has no negative attr, but this other one does -
why and what difference does it make?"

> 3) lets get these patches all into a patch series (one depends on 
> another) so that we can see them as a progression.
> 
> My experience touching large swaths of files in nova is that we very 
> much do want to break those up into chunks like cyeoh has already
> done, because the merge conflict issues abound on a patch set this
> big, if it's in a series it can slowly make its way in.

Agreed. Since we're keeping all the attributes (for now) I'll abandon
the current attribute replacement changesets but base new ones dependent
on 21492 (which they will be - the previous weren't actually dependent
on each other and so did not need to go in atomically or in any order).

The new ones won't have to go in atomically either, but smoke tests will
progressively get marked as smoke in the name of the test itself.

Regards,

Chris
-- 
cyeoh at au1.ibm.com


_______________________________________________
openstack-qa mailing list
openstack-qa at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa