Open Stack

On Tue, Apr 28, 2015 at 9:01 PM, Tom Fifield <tom at openstack.org> wrote:

> Hi all,
>
> I'm writing to you because I think we should seriously consider
> generating the API reference at least partially from the code itself,
> rather than continuing to maintain a separate set of documents that has
> no relationship with the code.
>

I definitely want to seriously consider autogenerating, but for the
reference portion only. It's important to bring up the quality of the API
docs by not purely autogenerating (I know you're not advocating for that)
but for strategically scraping. I like the ideas you have; more below.


>
> No-one can argue that it's an enormous amount of work to maintain an API
> reference. Despite our best efforts, our API reference is incomplete and
> inaccurate - you need only check the bug list[1] to know this.
>
> The move to swagger[2] will help some of this, but it's not enough. Even
> with new tooling, we are still stuck in a world of manual updates and
> effort - spending a large amount of time converting a python
> method/class definition into another format.
>
> We do add great value in the descriptions of parameters, groupings and
> overall display of the information. Diane is simply amazing here. So,
> wouldn't it be great if we could focus just on that, without needing to
> worry about changing this python docstring:
>
> """
> Move your app forward with the Uber API
> """
>
> into this swagger description:
>
> info:
>   description: Move your app forward with the Uber API
>
>
> and instead fix up the string itself ?
>
>
> The move-to-swagger spec at the moment says: "We don't really want
> autogenerated API documentation, because then how do you know if the
> code is correct?". Which is really quite odd - because looking at the
> code is exactly what developers, SDK makers and users are doing right
> now when confronted with our incomplete, inaccurate documentation :) The
> code is what defines the API - it's what actually processes the requests
> and responses. If there's a problem where the implementation of that
> processing is incorrect - we fix that in the code, not by telling users
> something completely different in a document :)
>

Well, we certainly won't solve the problem of inaccuracy by relying on
inaccurate code! But what I think will help is the API Working Group's
reviews and ensuring no backwards compatible changes get introduced. So the
tactic here is to ask the API WG to tackle the "contractual agreement"
portion of reviews that have APIImpact. Plus, keeping reviews with the core
teams will help immensely in keeping up with change.


>
> We may yet get to the point where OpenStack APIs are
> designed-in-document first, then implemented, but we're far from that at
> the moment and have much more pressing problems to solve.
>

Agreed, and we have years of data to back that up.


>
> There are a number of python integrations that swagger supports[3].
> Essentially, what these would allow us to do is use portions of the code
> to generate swagger specs. This means that our documentation would
> always be complete and accurate to what users actually experience when
> they interact with OpenStack APIs.


> The automation of option tables in the configuration reference is a good
> parallel here. We used to have very incomplete and often outdated
> configuration tables. After the automation, literally hundreds of bugs
> have been solved with very little human intervention. The problem just
> doesn't exist anymore, and we've clawed back some of our precious
> resources.
>
> In the config-ref there are two types of auto-magic: non-swift, where
> all text lives in the code and updates should be done there; and swift,
> where the option name comes from the code, but the descriptive text must
> be manually entered - overwriting a default label of "No help text
> available for this option".
>
> Applying that to API documents, I think it would be amazing to try the
> former: it basically means that developers write their own API
> documentation, happily, in the code, in a format they are familiar with.
> We can take that and generate swagger for the API-ref and display it
> appropriately.
>
> However, that might not be possible, or might be too much work for a
> single release. So, the second option is still a very good one - we use
> automation to extract some information (eg it could be just class
> titles, doc strings, parameter names) to generate swagger specs with the
> placeholder text that we-as-the-docs-team can fill in with our best
> efforts - which are excellent, since we're the pros at strings :)
>

I think this is a great first step, and I'd like to scope it to
infrastructure services only as well to keep the scope do-able.


>
> The important thing is that this should be done continuously[4], as with
> the config-ref, to ensure accuracy and completeness. Just looking at the
> first 75 bugs in api-site[5], this approach would solve almost all of
> them. This isn't something you can say about simply switching formats ;)
>

True. I do not want to switch formats only. I want to paint a picture of
the best future for this doc set. The overarching goals are:
- better process for keeping docs updated
- more trust in the accuracy of the API reference docs

I think by moving sourcing closer to the project repos themselves it'll
help with both goals. We'll just have to place accuracy and consumer
advocacy high in the review priority.

These new developer guides should enable app devs to make powerful apps on
OpenStack clouds. Those developer guides will be reviewed by the API WG,
the service devs themselves, and dev doc writers. Let's automate where it
makes sense to. The approach is not purely auto-generation -- that only
gets us halfway there. We also have to source docs where the best reviewers
live. We can insert narrative, helpful content that is heavily reviewed,
vetted, and offers high quality. The contract agreement burden can happen
at review time (for this iteration, we can tackle gated validation next if
we like).

Let's scrape code for resources, parameters, headers, requests and
responses, that'll definitely help. In doing so we can bake-in the
versioning of the API reference, does this run on icehouse? sorts of
questions.

I'll update the docs-spec, and I think I can get some great wireframes in
time for the Summit that show how awesome the future can be. I'll need web
dev, Python development, and time, but this is definitely shaping into
something do-able. Sweet.

I appreciate the API WG letting me discuss and shape with them at their
meeting this week also; feel free to read that log if you want to catch up.
[1]

Anne

1.
http://eavesdrop.openstack.org/meetings/api_wg/2015/api_wg.2015-04-30-00.01.log.html


>
> Anyway, thank you if you've managed to get to this point. I think we
> have a great opportunity here to make a very significant change to the
> way we do API documentation, and free up our previous docs resources to
> really focus on the things we do best, rather than spending a lot of
> time changing one arcane syntax into another :)
>
>
> Regards,
>
>
>
> Tom
>
> [1] https://bugs.launchpad.net/openstack-api-site
> [2] https://review.openstack.org/#/c/177934/
> [3] https://github.com/swagger-api/swagger-spec#python , I found there's
> a worked example showing the output of the swagger UI for
> django-rest-swagger at:
> http://django-rest-swagger.readthedocs.org/en/latest/examples.html
> [4] Kinda once a milestone is how we're doing it with config-ref at the
> moment, but this could in theory be fully automated
> [5] https://bugs.launchpad.net/openstack-api-site/+bugs?orderby=id&start=0
>
> -------- Forwarded Message --------
> Subject: [Blueprint autogenerate-api-reference] Autogenerate API Reference
> Date: Mon, 27 Apr 2015 15:14:41 -0000
> From: Anne Gentle <anne at openstack.org>
> Reply-To: Anne Gentle <anne at openstack.org>
> To: tom at openstack.org
>
> Blueprint changed by Anne Gentle:
>
> Whiteboard set to:
> We don't really want autogenerated API documentation, because then how
> do you know if the code is correct? How can a quality engineer or SDK
> dev trust the output? Autogeneration is fine for the first set of docs
> but after that need to be treated as the "contract" for SDK developers
> to trust.
>
> --
> Autogenerate API Reference
>
> https://blueprints.launchpad.net/openstack-manuals/+spec/autogenerate-api-reference
>
>
>
> _______________________________________________
> OpenStack-docs mailing list
> OpenStack-docs at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-docs
>



-- 
Anne Gentle
annegentle at justwriteclick.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-docs/attachments/20150501/243cf994/attachment-0001.html>