Open Stack

On Mon, 12 Jan 2015, Anne Gentle wrote:

> I'm interested, sure. What did you use to write the HTTP tests, as in, what
> was the source of truth for what the requests and responses should be?

That is an _extremely_ good question and one I really struggled with
as I started integrating gabbi with ceilometer.

Initially I thought "I'll just use the API docs[1] as the source of
truth" but I found they were a bit incomplete on some of the nuances,
so I asked around for other sources of truth, but got little in the
way of response.

So then I tried to use the api controller code, but not to put too fine a
point on it, but the combination of WSME and Pecan makes for utterly
inscrutable code if you're interested in the actual structure of the
HTTP requests and response and the URIs being used.

So then I tried to use the existing api unit tests and was able to
extract a bit there, but it wasn't smooth sailing.

So finally what I did was decide that I would do the work in phases
and with collaborators: I'd get the initial framework in place and
then impose upon those more familiar with the API than I to do
subsequent dependent patchsets that cover the API more completely.

I have to admit that the concept of API truth is part of the reason I
wanted to create this kind of testing. None of the resources I could
find in the ceilometer code tree gave any clear overview that mapped
URIs to methods, allowing easy discovery of how the code works. I
wanted to find some kind of map[2]. Gabbi itself doesn't solve this
problem (there's no map between URI and python method) but it can at
least show the API, there in the code. It's a step in the right
direction.

I know that there are discussions in progress about formalizing APIs
with tools like RAML (for example the thread Ian just extended[3]). I
think these have their place, especially for declaring truth, but they
aren't necessarily good learning aids for new developers or good
assistants for enabling and maintaining transparency.

[1] I started at: http://docs.openstack.org/developer/ceilometer/webapi/v2.html
but I think I should have used: http://developer.openstack.org/api-ref-telemetry-v2.html

[2] https://github.com/tiddlyweb/tiddlyweb/blob/master/tiddlyweb/urls.map

[3] http://lists.openstack.org/pipermail/openstack-dev/2015-January/054153.html
-- 
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent