Open Stack

Hi all,

Starting a thread to discuss $subject, as requested in:

https://review.openstack.org/#/c/51558/

First a bit of background.  I wrote a keystoneclient patch, and ayoung
stated he'd like it tested via tempest before he'd ack it:

https://review.openstack.org/#/c/48462/

So I spoke to ayoung and dkranz on IRC, showing them my local tests for the
patch.  dkranz suggested creating a "client_lib" directory, where we could
build out a more comprehensive set of tests over time, adding to the inital
tests related to keystone trusts client additions.

A couple of things to note:
- These are end-to-end tests, designed to test not only the client, but
  also the API and keystone backend functionality.  So arguably this could
  just be a scenario test, e.g scenario/keystone/test_v3_auth.py

- The intention is to excercise logic which is hard to fully test with
  unit or integration tests, and to catch issues like incompatibility
  between client and API - e.g keystoneclient tests may pass, but we need
  to make sure the client actually works against the real keystone API.

Working on Heat has given me a pretty good insight into the python-*client
API's, as we use them to orchestrate actions with every openstack service;
IMO anything we can do to make these interfaces more robust (and catch
bugs, several of which I found already while writing these tests) is a
good-thing (tm).

I'd welcome feedback on the patch above, and what will be the most
acceptable approach to the tempest team for adding these tests.

More links:

https://review.openstack.org/#/c/51559/
https://review.openstack.org/#/c/51560/
https://blueprints.launchpad.net/tempest/+spec/keystoneclient-api

Thanks!

Steve