[openstack-dev] [qa][keystone] Keystoneclient tests to tempest
mordred at inaugust.com
Sun Dec 8 23:20:29 UTC 2013
Thanks - I've been wanting to kill this for a long time. Thanks for
starting the discussion...
On 12/08/2013 07:26 PM, Brant Knudson wrote:
> We'd like to get the keystoneclient tests out of keystone. They're
> serving a useful purpose of catching problems with non-backwards
> compatible changes in keystoneclient so we still want them run. Problem
> is they're running at the wrong time -- only on changes to keystone and
> not changes to keystoneclient.
> The tests need to be run:
> When keystoneclient changes
> - run the tests against the change
> When the tests change
> - run the change against the current keystoneclient and also old clients
> When keystone changes
> - run the tests against the change with current client
You're talking about this:
Which is almost ready to go.
> So here's what I think we need to do to get keystone client tests out of
> 1) Figure out where to put the tests - is it tempest or something else?
Tempest. They should be moved to tempest.
> 2) Write up a test and put it there
> 3) Have a job that when there's a change in the tests it runs against
> current client lib
Don't need most of that - the generalized "test clients against stable
versions" matrix is about to land - as long as tempest has your tests in
the scenarios, you'll be fine.
> 4) Expand the job to also run against old clients
> - or is there 1 job per version?
> - what versions? (keystone does master, essex-3, and 0.1.1)
> - e.g. tox -e master,essex-3,0.1.1
essex and 0.1.1 are both older than dirt. We just removed folsom from
the gate. I'd got ahead and remove these.
> - suggest start with these versions and then consider what to use in
OpenStack has a clear support policy - the gate infrastructure will be
covering that. Done!
> 5) Now we can start adding tests
> 6) Have a job that when there's a change in keystoneclient it runs
> these tests against the change
> 7) When there's a change in keystone, run these tests against the change
Yup. Already accounted for.
> 8) Copy the keystoneclient tests from keystone to the new location --
> will require some changes
> 9) Remove the tests from keystone \o/
> 10) Move tests back to keystone where makes sense -- use webtest like v3
> I created an etherpad with this same info so it's easier to discuss:
> - Brant
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
More information about the OpenStack-dev