[openstack-dev] [qa][keystone] Keystoneclient tests to tempest

Brant Knudson blk at acm.org
Mon Dec 9 15:12:19 UTC 2013


Monty -

Thanks for doing the work already to get the infrastructure set up. Looks
like I've got the easy part here. I posted an initial patch that has one
test from keystone in https://review.openstack.org/#/c/60724/ . I hope to
be able to move all the tests over unchanged. The tricky part is getting
all the fixtures set up the same way that keystone does.

I'm not sure what directory the tests go in Tempest. Maybe it's a new one
like shardy's attempts: https://review.openstack.org/#/c/54810/ . I'll
leave that to the tempest team.

- Brant

On Sun, Dec 8, 2013 at 5:20 PM, Monty Taylor <mordred at inaugust.com> wrote:

> Hi!
>
> Thanks - I've been wanting to kill this for a long time. Thanks for
> starting the discussion...
>
> On 12/08/2013 07:26 PM, Brant Knudson wrote:
> >
> > We'd like to get the keystoneclient tests out of keystone. They're
> > serving a useful purpose of catching problems with non-backwards
> > compatible changes in keystoneclient so we still want them run. Problem
> > is they're running at the wrong time -- only on changes to keystone and
> > not changes to keystoneclient.
> >
> > The tests need to be run:
> >
> > When keystoneclient changes
> >  - run the tests against the change
> >
> > When the tests change
> >  - run the change against the current keystoneclient and also old clients
> >
> > When keystone changes
> >  - run the tests against the change with current client
>
> You're talking about this:
>
> https://review.openstack.org/#/c/41931/
>
> Which is almost ready to go.
>
>
> So here's what I think we need to do to get keystone client tests out of
> > keystone:
> >
> >  1) Figure out where to put the tests - is it tempest or something else?
>
> Tempest. They should be moved to tempest.
>
> >  2) Write up a test and put it there
> >  3) Have a job that when there's a change in the tests it runs against
> > current client lib
>
> Don't need most of that - the generalized "test clients against stable
> versions" matrix is about to land - as long as tempest has your tests in
> the scenarios, you'll be fine.
>
> >  4) Expand the job to also run against old clients
> >     - or is there 1 job per version?
> >     - what versions? (keystone does master, essex-3, and 0.1.1)
> >     - e.g. tox -e master,essex-3,0.1.1
>
> essex and 0.1.1 are both older than dirt. We just removed folsom from
> the gate. I'd got ahead and remove these.
>
> >     - suggest start with these versions and then consider what to use in
> > future
>
> OpenStack has a clear support policy - the gate infrastructure will be
> covering that. Done!
>
>
What is the support policy for the client libs? Nobody's tried to remove
anything since I started reviewing.

>  5) Now we can start adding tests
>
> Tempest scenarios.
>
> >  6) Have a job that when there's a change in keystoneclient it runs
> > these tests against the change
> >  7) When there's a change in keystone, run these tests against the change
>
> Yup. Already accounted for.
>
> >  8) Copy the keystoneclient tests from keystone to the new location --
> > will require some changes
> >  9) Remove the tests from keystone \o/
> > 10) Move tests back to keystone where makes sense -- use webtest like v3
> > tests
> >
> > I created an etherpad with this same info so it's easier to discuss:
> > https://etherpad.openstack.org/p/KeystoneTestsToTempest
> >
> > - Brant
> >
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131209/2993e4de/attachment.html>


More information about the OpenStack-dev mailing list