Open Stack

On 07/08/2013 08:22 AM, Anne Gentle wrote:
> More below.
>
> Anne Gentle
> Content Stacker
> anne at openstack.org
>
>
> On Jul 8, 2013, at 8:00 AM, Sean Dague <sean at dague.net> wrote:
>
>> On 07/02/2013 01:54 PM, David Kranz wrote:
>>> On 07/02/2013 12:58 PM, Maru Newby wrote:
>>>> On Jul 2, 2013, at 11:35 AM, Daryl Walleck
>>>> <daryl.walleck at RACKSPACE.COM> wrote:
>>>>
>>>>> I don't see a link between testing and being able to generate a
>>>>> client from documents. The time to create/maintain the clients for me
>>>>> at least has been negligible in comparison to the time spent writing
>>>>> tests and interfaces with other critical parts of the OpenStack
>>>>> infrastructure I want to test. I do think an architectural discussion
>>>>> of the clients in worthwhile (for what it's worth, the json/xml
>>>>> client structure was based on the code I originally submitted for
>>>>> Tempest and even I hate the concept now), but at some point we need
>>>>> to move forward and talk about the lower level details of OpenStack
>>>>> that we're not testing and developing interfaces for that.
>>>> The point is to remove mechanical effort, not just be able to generate
>>>> clients.  The majority of api tests could be generated too - including
>>>> serialization-specific validation and fuzz testing - if sufficiently
>>>> detailed api specs were available.  The api clients themselves could
>>>> be generated from those same specs.  And then, we'd have more cycles
>>>> to test the truly interesting stuff.
>>>>
>>>>
>>>> m.
>>> I think Maru is right, at least in theory. There existed some kind of
>>> xml spec for compute, at least at Rackspace. Don't know the current
>>> status of that, and support for extensions, or the plan for the new api.
>>> Most other projects don't seem to bother. But look at (mostly a
>>> skeleton) https://wiki.openstack.org/wiki/Neutron/API/WADL.
>>
>> In the awesome world that is awesome, where the API is self documenting, with a standards based way to auto generating client code, things are sure awesome.
>>
>
> We are having a conversation about this on the Openstack-docs mailing list. It looks like there's a plan for requests and responses but not a spec.
>
>> But the reality is that there really is nothing like that in REST universion. WADL was a proposed spec, that's basically died on the vine, with Sun. Most OpenStack APIs provide a version of it, in Nova's case *non* of the extensions are specified in it. Also, it's often wrong, because no one uses it.
>>
>
> Each extension is documented in a separate WADL.
>
>> There also aren't any good tools around WADL (again, because it's a dead spec), so we'd be writing our own generators instead of writing our own clients. Doing so for a dead spec doesn't seem like a huge win there.
>>
>> I do actually agree that documenting the API and testing the API are actually very closely related. But the reality is for most of our API layers we aren't there yet. Also auto generated tests are probably only useful for the negative side anyway, as the positive side relies on side effects, and those aren't machine codable in specs anyway.
>
> Aw I'm sad to hear you throw in the towel on docs and testing being together. What has led to this?

Mostly just time, and the fact that across 9 core services there is very 
little commonality on how this is done. So it's a bigger effort than the 
QA team can manage to wrangle that all together.

That being said, if someone else wants to jump on this, cool. It would 
be a cool place to be, but it requires a lot more cross project work 
than I think anyone can currently manage.

	-Sean

-- 
Sean Dague
http://dague.net