Open Stack

We'd like to get the keystoneclient tests out of keystone. They're serving
a useful purpose of catching problems with non-backwards compatible changes
in keystoneclient so we still want them run. Problem is they're running at
the wrong time -- only on changes to keystone and not changes to
keystoneclient.

The tests need to be run:

When keystoneclient changes
 - run the tests against the change

When the tests change
 - run the change against the current keystoneclient and also old clients

When keystone changes
 - run the tests against the change with current client

So here's what I think we need to do to get keystone client tests out of
keystone:

 1) Figure out where to put the tests - is it tempest or something else?
 2) Write up a test and put it there
 3) Have a job that when there's a change in the tests it runs against
current client lib
 4) Expand the job to also run against old clients
    - or is there 1 job per version?
    - what versions? (keystone does master, essex-3, and 0.1.1)
    - e.g. tox -e master,essex-3,0.1.1
    - suggest start with these versions and then consider what to use in
future
 5) Now we can start adding tests
 6) Have a job that when there's a change in keystoneclient it runs these
tests against the change
 7) When there's a change in keystone, run these tests against the change
 8) Copy the keystoneclient tests from keystone to the new location -- will
require some changes
 9) Remove the tests from keystone \o/
10) Move tests back to keystone where makes sense -- use webtest like v3
tests

I created an etherpad with this same info so it's easier to discuss:
https://etherpad.openstack.org/p/KeystoneTestsToTempest

- Brant
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131208/f7c25384/attachment.html>