Open Stack

Mehdi did some interesting work to explore integrating tempest
and gabbi in a tempest plugin for gnocchi[1]. Inspired by that I've
started exploring more generic ways of doing gabbi+tempest.

That work is at[2]. I'm posting about it for a couple reasons:

* I'm not super savvy about tempest so I could be missing out on
   ways to do things correctly; input most appreciated.

* As soon as I started writing tests in the gabbi style against both
   Nova and Glance I found (and reported[3]) multiple HTTP-related bugs (for
   example, the glance discovery resource returns links which 404).

   This suggests that having tooling which makes it pretty easy
   to dynamically write and run tests against OpenStack without
   assumption-having clients is a good idea.

For those that don't know, gabbi[4] is a testing harness for writing
HTTP requests and expected responses in a YAML format with
(optional) JSONPath evaluation of responses.

It is particularly useful in the OpenStack context because you can use
it to write tests from the standpoint of an uninformed client: write
requests that might make sense and see what happens, discover bugs
and behavior that violates the principle of least surprise, easily.

Most OpenStack API tests don't have this standpoint so frequently
don't explore edge cases that any curious real client might very
well do.

Anyway, I hope this is useful to people in some fashion. Ideally
over the long run it would be useful to turn this into a library
that other tempest plugins can use (instead of being a plugin
itself). If that sounds interesting or you have ideas, let me know.

Thanks.

[1] https://review.openstack.org/#/c/301585/
[2] https://github.com/cdent/gabbi-tempest
[3] https://bugs.launchpad.net/nova/+bugs?field.tag=gabbi
[4] https://gabbi.readthedocs.org/


-- 
Chris Dent               (¨s¡ã¡õ¡ã)¨s¦à©ß©¥©ß            http://anticdent.org/
freenode: cdent                                         tw: @anticdent