Open Stack

On Mon, Jul 8, 2013 at 8:03 AM, Sean Dague <sean at dague.net> wrote:

> On 07/08/2013 08:22 AM, Anne Gentle wrote:
>
>> More below.
>>
>> Anne Gentle
>> Content Stacker
>> anne at openstack.org
>>
>>
>> On Jul 8, 2013, at 8:00 AM, Sean Dague <sean at dague.net> wrote:
>>
>>  On 07/02/2013 01:54 PM, David Kranz wrote:
>>>
>>>> On 07/02/2013 12:58 PM, Maru Newby wrote:
>>>>
>>>>> On Jul 2, 2013, at 11:35 AM, Daryl Walleck
>>>>> <daryl.walleck at RACKSPACE.COM> wrote:
>>>>>
>>>>>  I don't see a link between testing and being able to generate a
>>>>>> client from documents. The time to create/maintain the clients for me
>>>>>> at least has been negligible in comparison to the time spent writing
>>>>>> tests and interfaces with other critical parts of the OpenStack
>>>>>> infrastructure I want to test. I do think an architectural discussion
>>>>>> of the clients in worthwhile (for what it's worth, the json/xml
>>>>>> client structure was based on the code I originally submitted for
>>>>>> Tempest and even I hate the concept now), but at some point we need
>>>>>> to move forward and talk about the lower level details of OpenStack
>>>>>> that we're not testing and developing interfaces for that.
>>>>>>
>>>>> The point is to remove mechanical effort, not just be able to generate
>>>>> clients.  The majority of api tests could be generated too - including
>>>>> serialization-specific validation and fuzz testing - if sufficiently
>>>>> detailed api specs were available.  The api clients themselves could
>>>>> be generated from those same specs.  And then, we'd have more cycles
>>>>> to test the truly interesting stuff.
>>>>>
>>>>>
>>>>> m.
>>>>>
>>>> I think Maru is right, at least in theory. There existed some kind of
>>>> xml spec for compute, at least at Rackspace. Don't know the current
>>>> status of that, and support for extensions, or the plan for the new api.
>>>> Most other projects don't seem to bother. But look at (mostly a
>>>> skeleton) https://wiki.openstack.org/**wiki/Neutron/API/WADL<https://wiki.openstack.org/wiki/Neutron/API/WADL>
>>>> .
>>>>
>>>
>>> In the awesome world that is awesome, where the API is self documenting,
>>> with a standards based way to auto generating client code, things are sure
>>> awesome.
>>>
>>>
>> We are having a conversation about this on the Openstack-docs mailing
>> list. It looks like there's a plan for requests and responses but not a
>> spec.
>>
>>  But the reality is that there really is nothing like that in REST
>>> universion. WADL was a proposed spec, that's basically died on the vine,
>>> with Sun. Most OpenStack APIs provide a version of it, in Nova's case *non*
>>> of the extensions are specified in it. Also, it's often wrong, because no
>>> one uses it.
>>>
>>>
>> Each extension is documented in a separate WADL.
>>
>>  There also aren't any good tools around WADL (again, because it's a dead
>>> spec), so we'd be writing our own generators instead of writing our own
>>> clients. Doing so for a dead spec doesn't seem like a huge win there.
>>>
>>> I do actually agree that documenting the API and testing the API are
>>> actually very closely related. But the reality is for most of our API
>>> layers we aren't there yet. Also auto generated tests are probably only
>>> useful for the negative side anyway, as the positive side relies on side
>>> effects, and those aren't machine codable in specs anyway.
>>>
>>
>> Aw I'm sad to hear you throw in the towel on docs and testing being
>> together. What has led to this?
>>
>
> Mostly just time, and the fact that across 9 core services there is very
> little commonality on how this is done. So it's a bigger effort than the QA
> team can manage to wrangle that all together.
>
>
Yeah I'm definitely feeling that way on the docs side too. How can we do a
collective bargaining agreement with dev to ask them to take on some of
these goals? :)


> That being said, if someone else wants to jump on this, cool. It would be
> a cool place to be, but it requires a lot more cross project work than I
> think anyone can currently manage.
>
>
>         -Sean
>
> --
> Sean Dague
> http://dague.net
>
> ______________________________**_________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.**org <openstack-qa at lists.openstack.org>
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-qa<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa>
>



-- 
Anne Gentle
annegentle at justwriteclick.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-qa/attachments/20130708/529ee561/attachment.html>