Open Stack

On 29 June 2015 at 11:30, Anne Gentle <annegentle at justwriteclick.com> wrote:

>
> On Sat, Jun 27, 2015 at 10:43 PM, Russell Sim <russell.sim at gmail.com>
> wrote:
>
>>
>> Raw swaggerish files http://fairy-slipper.russellsim.org/ you can copy
>> and paste the urls into the swagger UI's above to view them.
>>
>> I have picked up some inconsistencies in the WADL files that I have been
>> gradually fixing these. The following 2 outstanding reviews are all that's
>> needed to fix things well enough to make the dumb converter that I've
>> written work.
>> https://review.openstack.org/#/c/195364/
>> https://review.openstack.org/#/c/196407/
>>
>>
> Both merged, cool.
>

Yep, i must say that ppl are very responsive with reviews :)  many thanks
to the docs team!


> I keep saying Swaggerish, because the swagger output isn't actually
>> swagger, it's modified to support multiple request/response types at the
>> same URL.  Basically, because the OpenStack API isn't really REST. :(
>>
>>
> Heh, it's a lot to document any way you look at it.
>
> But it's cool you got the multiple request/response worked out. An example
> for those who are reading along is the Orchestration actions resource:
>
> http://developer.openstack.org/api-ref-orchestration-v1.html#stack_action_suspend
>
>
> http://developer.openstack.org/api-ref-orchestration-v1.html#stack_action_resume
>

The worst offender is the
http://developer.openstack.org/api-ref-compute-v2.html#compute_server-actions

But there are more subtle examples.  Tom assigned me bug
https://bugs.launchpad.net/openstack-manuals/+bug/1338131 it's about
confusing documentation of request and response object. So where a request
may take several different request objects and return different response
objects. Perhaps the keystone v3 token url should actually just be
represented like the actions endpoint.  So one request object token per
auth type?

The code repository I'm using is https://github.com/russell/fairy-slipper
>> but beware, the code is a disgrace.  I tried using a proper WADL parser,
>> but I couldn't get it to work, so instead I have a horrid SAX based parser
>> that drunkenly fumbles through the files trying really hard not to crash.
>>
>> What's next.
>> - Improve the Docbok/WADL parsers, so that it picks up the methods that
>> are currently missing.
>>
>
> I want to help with this, is the best way to side-by-side compare the
> output with the current WADL (or HTML output from WADL)?
>

Funny you should ask so there are actually a number of minor problem which
the parser has picked up in the existing WADL.

http://fairy-slipper.russellsim.org/logs/wadl_to_swagger.log

There are some of false positivities, because a WADL file may be getting
used my multiple doc book files.  I'll look at cleaning it up tonight.

But the log reads like, parses json docbook description, then parses each
referenced WADL file.

2015-06-29 02:14:23,545 /opt/fairy-slipper/tools/wadl_to_swagger.py
INFO Reading API description from
/var/www/html/api-ref/api-ref-orchestration-v1.json
2015-06-29 02:14:23,546 /opt/fairy-slipper/tools/wadl_to_swagger.py
INFO Parsing /opt/api-site/api-ref/src/wadls/orchestration-api/src/v1/orchestration-api.wadl
2015-06-29 02:14:23,589 /opt/fairy-slipper/tools/wadl_to_swagger.py
WARNING No tags for method resource_type_list
2015-06-29 02:14:23,589 /opt/fairy-slipper/tools/wadl_to_swagger.py
WARNING No tags for method resource_type_get
2015-06-29 02:14:23,589 /opt/fairy-slipper/tools/wadl_to_swagger.py
WARNING No tags for method resource_type_template

If i were a betting man I would think that there is no documentation for
resource type list/get/template in
http://developer.openstack.org/api-ref-orchestration-v1.html What has
happened is that the docbook file doesn't reference these methods.  But
someone did actually document the methods in the WADL.

Any service with extensions will probably suffer from the false positives,
but as another simple example.

2015-06-29 02:37:42,065 /opt/fairy-slipper/tools/wadl_to_swagger.py
INFO Reading API description from
/var/www/html/api-ref/api-ref-image-v1.json
2015-06-29 02:37:42,086 /opt/fairy-slipper/tools/wadl_to_swagger.py
INFO Parsing /opt/api-site/api-ref/src/wadls/image-api/src/v1/os-image-1.0.wadl
2015-06-29 02:37:42,110 /opt/fairy-slipper/tools/wadl_to_swagger.py
WARNING No tags for method listImage-v1

Sure enough there is only the list image detailed documented in
http://developer.openstack.org/api-ref-image-v1.html

You can ignore,  lines like  "Can't find method setUserEnabled" i have
patches for both of those but i have only created a review for one.
https://review.openstack.org/#/c/194859/

Any method that is missing tags will not be ported into Swagger at the
moment.


In regards to helping with checking the conversion.  I have started
creating ReST output http://fairy-slipper.russellsim.org/rst/ that would be
the best thing to check against.  I would look at the html doc and compare
it to the ReST.  That should be the easiest way to confirm that things are
being translated correctly, return status codes and other bits are still
missing, but the main method documentation including formatting is there.
I'm not converting the chapter documentation to ReST, but I'll get that
done tonight.


>
>> - Convert Swaggerish into ReST/JSONSchema (started, but nowhere near
>> complete)
>>
>
> Where should the JSON Schema live, in the fairy-slipper repo for now??
>

My thoughts are that we keep everything in fairy-slipper, organised
appropriately by service.  We can then submit reviews to get it included
into the projects that are willing.  As it gets migrated to the services,
we can remove it from fairy-slipper and pull it from the service API
directly.  What that actually means in practice isn't decided, but those
are my thoughts.


 - Write webserver to render ReST/JSONSchema to Swaggerish (started, but
> nowhere near complete)
>
> Anyone interested in this one?
>

I'm actually happy to carry the weight.  I would rather get a framework of
a service completed before asking people to contribute.  Not that i have
any super cool ideas, but at the same time like any programmer I'm
massively opinionated about 'the right way' to do things.  (Like that term
has any real meaning) But the primary concern is consistency.  I think I
would be better if i can try to get things into a reasonably consistent
state where we can extend in a consistent way.

Also, I see that as the fun part :D


>
>
>> - Update Swagger UI to include Tag documentation. (currently swagger has
>> no understanding of the documentation at the top of each chapter.  It's
>> included in the swaggerish files but it's not rendered in the client
>> presently)
>>
>>
> I'd like to pull out the descriptive info into RST files or some such. I'm
> guessing it's text like what's in the Servers info here?
> http://developer.openstack.org/api-ref-compute-v2.html#compute_servers
>
If it's something else, let me know.
>

Ah, glad you asked that is exactly the documentation that is included in a
Tag.  When you view the demos above each of the API endpoints are grouped
by Tag.  These tags don't support documentation in Swagger by default.  But
i'm including it in the schema so we can reproduce exactly the
documentation you mention above.  At the conference ppl mentioned narrative
documentation, is this what that term describes?

The other thing to consider is that the Tag concept is reasonably
flexible.  We could have a method in multiple Tags?  We could develop a Tag
hierarchy with a naming scheme like:

server
server::actions
server::metadata
server::actions

We could then perhaps develop a better way of browsing the API using the
Tags?

I originally had really high hopes of converting the doc and pushing it
>> into the OpenStack services, I've scoped this back a bit initially because
>> I think it would be better to convert the current documentation into the
>> new formats first.  We will need a way to host this on the openstack.org
>> website which may not have access to a running OpenStack deployment.  Once
>> I have this in a form that is acceptable, then I'll look at how this will
>> be best integrated the documentation into each of the OpenStack services.
>>
>
> Yep, I'm good with this approach. Should we patch the openstack/api-site
> repo in the interim for testing purposes? Or just leave it in fairy-slipper
> while we keep working?
>

My thoughts are that we keep the WADL for the time being.  Once we have
basically a viable solution to replace the current documentation then I
will bundle it up as a review?  The other option would be once the proof of
concept is usable as a complete system?  At the moment things are in quite
a state of flux.

I'll update the spec at https://review.openstack.org/#/c/177934/ and merge
> it pretty quickly as I think the approach is sound and we should start
> dividing the work as we can.
>
> Thanks so much for working on this! I know we can share some tasks so it
> doesn't feel like you're eating an elephant!
>

I realise that i'm probably coming across as quite conservative.  This is
not really true, I am concerned about having other people work on this
while there are no tests and there is no real structure or defined way to
generate the files.  I guess once it's reasonably automated (gimme 1 more
week :))  then we can probably look at how others can help extend and
develop the functionality.  But at the moment I don't even really have the
skeleton walking.  It's about half way there, perhaps 1/3.  But what we
should do is plan the things that aren't yet defined.

So yes I am keen to have other people contribute code, but just not with
code this very second.

I am curious about how the migration will work:
When this is proposed as a patch will we delete the WADL files immediately?
 or will they hang around?
Perhaps we could use a feature branch?  That way the OPs team can have a
chance to prep the openstack.org website for the transition and we can
maintain the WADL files until we are ready to launch?

What are we going to do about the frontend?  That is an area that will
require some thought about how to present the information.  I personally
don't like either example above.  I don't like the current API doc, too
much scrolling, no navigation.  jensoleg's is crap because of too much
scrolling, also features confusing amounts of scrolling for large response
examples.  The default swagger interface sucks because it assumes I am a
robot?  why is the URL more important that what the URL does?  Basically i
think we can do better.  I think this is probably the best interface for
API documentation i have seen so far http://docs.apimatic.apiary.io/ but if
people can suggest other sites we can draw ideas from that would be great.

-- 
Cheers,
Russell Sim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-docs/attachments/20150629/6a15fcc6/attachment-0001.html>