Open Stack

On 07/02/2013 01:54 PM, David Kranz wrote:
> On 07/02/2013 12:58 PM, Maru Newby wrote:
>> On Jul 2, 2013, at 11:35 AM, Daryl Walleck
>> <daryl.walleck at RACKSPACE.COM> wrote:
>>
>>> I don't see a link between testing and being able to generate a
>>> client from documents. The time to create/maintain the clients for me
>>> at least has been negligible in comparison to the time spent writing
>>> tests and interfaces with other critical parts of the OpenStack
>>> infrastructure I want to test. I do think an architectural discussion
>>> of the clients in worthwhile (for what it's worth, the json/xml
>>> client structure was based on the code I originally submitted for
>>> Tempest and even I hate the concept now), but at some point we need
>>> to move forward and talk about the lower level details of OpenStack
>>> that we're not testing and developing interfaces for that.
>> The point is to remove mechanical effort, not just be able to generate
>> clients.  The majority of api tests could be generated too - including
>> serialization-specific validation and fuzz testing - if sufficiently
>> detailed api specs were available.  The api clients themselves could
>> be generated from those same specs.  And then, we'd have more cycles
>> to test the truly interesting stuff.
>>
>>
>> m.
> I think Maru is right, at least in theory. There existed some kind of
> xml spec for compute, at least at Rackspace. Don't know the current
> status of that, and support for extensions, or the plan for the new api.
> Most other projects don't seem to bother. But look at (mostly a
> skeleton) https://wiki.openstack.org/wiki/Neutron/API/WADL.

In the awesome world that is awesome, where the API is self documenting, 
with a standards based way to auto generating client code, things are 
sure awesome.

But the reality is that there really is nothing like that in REST 
universion. WADL was a proposed spec, that's basically died on the vine, 
with Sun. Most OpenStack APIs provide a version of it, in Nova's case 
*non* of the extensions are specified in it. Also, it's often wrong, 
because no one uses it.

There also aren't any good tools around WADL (again, because it's a dead 
spec), so we'd be writing our own generators instead of writing our own 
clients. Doing so for a dead spec doesn't seem like a huge win there.

I do actually agree that documenting the API and testing the API are 
actually very closely related. But the reality is for most of our API 
layers we aren't there yet. Also auto generated tests are probably only 
useful for the negative side anyway, as the positive side relies on side 
effects, and those aren't machine codable in specs anyway.

	-Sean

-- 
Sean Dague
http://dague.net