<div dir="ltr"><br><div class="gmail_extra"><div class="gmail_quote">On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy <span dir="ltr"><<a href="mailto:shardy@redhat.com" target="_blank">shardy@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hi all,<br>
<br>
Starting a thread to discuss $subject, as requested in:<br>
<br>
<a href="https://review.openstack.org/#/c/51558/" target="_blank">https://review.openstack.org/#/c/51558/</a><br>
<br>
First a bit of background. I wrote a keystoneclient patch, and ayoung<br>
stated he'd like it tested via tempest before he'd ack it:<br>
<br>
<a href="https://review.openstack.org/#/c/48462/" target="_blank">https://review.openstack.org/#/c/48462/</a><br>
<br>
So I spoke to ayoung and dkranz on IRC, showing them my local tests for the<br>
patch. dkranz suggested creating a "client_lib" directory, where we could<br>
build out a more comprehensive set of tests over time, adding to the inital<br>
tests related to keystone trusts client additions.<br>
<br>
A couple of things to note:<br>
- These are end-to-end tests, designed to test not only the client, but<br>
also the API and keystone backend functionality. So arguably this could<br>
just be a scenario test, e.g scenario/keystone/test_v3_auth.py<br></blockquote><div><br></div><div>I'd love to be able to run these tests against a wider variety of service configurations (e.g. LDAP!), which tempest is obviously more suitable for.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
- The intention is to excercise logic which is hard to fully test with<br>
unit or integration tests, and to catch issues like incompatibility<br>
between client and API - e.g keystoneclient tests may pass, but we need<br>
to make sure the client actually works against the real keystone API.<br></blockquote><div><br></div><div>All of our tests under keystone.tests.test_keystoneclient fall into this category as well:</div><div><br></div><div>
<a href="https://github.com/openstack/keystone/blob/a0e26c1882d83989bee3726a5ae08cbe3f32a2b5/keystone/tests/test_keystoneclient.py">https://github.com/openstack/keystone/blob/a0e26c1882d83989bee3726a5ae08cbe3f32a2b5/keystone/tests/test_keystoneclient.py</a></div>
<div><br></div><div> <a href="https://github.com/openstack/keystone/blob/a0e26c1882d83989bee3726a5ae08cbe3f32a2b5/keystone/tests/test_keystoneclient_sql.py">https://github.com/openstack/keystone/blob/a0e26c1882d83989bee3726a5ae08cbe3f32a2b5/keystone/tests/test_keystoneclient_sql.py</a></div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Working on Heat has given me a pretty good insight into the python-*client<br>
API's, as we use them to orchestrate actions with every openstack service;<br>
IMO anything we can do to make these interfaces more robust (and catch<br>
bugs, several of which I found already while writing these tests) is a<br>
good-thing (tm).<br></blockquote><div><br></div><div>++</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
I'd welcome feedback on the patch above, and what will be the most<br>
acceptable approach to the tempest team for adding these tests.<br>
<br>
More links:<br>
<br>
<a href="https://review.openstack.org/#/c/51559/" target="_blank">https://review.openstack.org/#/c/51559/</a><br>
<a href="https://review.openstack.org/#/c/51560/" target="_blank">https://review.openstack.org/#/c/51560/</a><br>
<a href="https://blueprints.launchpad.net/tempest/+spec/keystoneclient-api" target="_blank">https://blueprints.launchpad.net/tempest/+spec/keystoneclient-api</a><br>
<br>
Thanks!<br>
<br>
Steve<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div><br></div>-Dolph
</div></div>