Open Stack

hi all, i missed the original email due to not being on this list so i'm 
going to copy-paste and quote inline.

firstly, thanks for bringing this up Tom.

> I'm writing to you because I think we should seriously consider
> generating the API reference at least partially from the code itself,
> rather than continuing to maintain a separate set of documents that has
> no relationship with the code.

i agree, i think it would be great if projects could start from an 
autogenerated base for their api-ref.


> No-one can argue that it's an enormous amount of work to maintain an API
> reference. Despite our best efforts, our API reference is incomplete and
> inaccurate - you need only check the bug list[1] to know this.
>
> The move to swagger[2] will help some of this, but it's not enough. Even
> with new tooling, we are still stuck in a world of manual updates and
> effort - spending a large amount of time converting a python
> method/class definition into another format.

i'd love to see the density of workload shift slightly more on to the 
project teams for the api-ref doc side of things. maybe it will help 
ease some of the manual updating pressure on the doc team.

the sahara team recently did their api-ref wadls and i know it was a 
hard fought battle with several of us participating. also big props to 
Diane for helping us through that process.


> We do add great value in the descriptions of parameters, groupings and
> overall display of the information. Diane is simply amazing here. So,
> wouldn't it be great if we could focus just on that, without needing to
> worry about changing this python docstring:
>
> """
> Move your app forward with the Uber API
> """
>
> into this swagger description:
>
> info:
>   description: Move your app forward with the Uber API
>
>
> and instead fix up the string itself ?

i'm a big fan of increasing the usability of the docstrings to become 
more informative. i realize this might be a slightly selfish desire as i 
enjoy seeing them in the code, but i think it could pay off well if we 
were to harvest the documentation to create something useful for users.

i would love to see some sort of oslo-ish package that would allow us to 
have a neutral format in the docstrings that could then be transformed 
into swagger or another format if necessary.


> The move-to-swagger spec at the moment says: "We don't really want
> autogenerated API documentation, because then how do you know if the
> code is correct?". Which is really quite odd - because looking at the
> code is exactly what developers, SDK makers and users are doing right
> now when confronted with our incomplete, inaccurate documentation :) The
> code is what defines the API - it's what actually processes the requests
> and responses. If there's a problem where the implementation of that
> processing is incorrect - we fix that in the code, not by telling users
> something completely different in a document :)

+1, improving documentation in the code is a noble effort =)


> We may yet get to the point where OpenStack APIs are
> designed-in-document first, then implemented, but we're far from that at
> the moment and have much more pressing problems to solve.

i'm completely intrigued by this process and i love what swagger is 
proposing/doing in this problem space. i'm unsure that we would ever 
reach this point with openstack projects but i think it would provide 
interesting options for new projects if we had some sort of path for 
this type of development.

i can imagine a world where a new project is able to get their wsgi 
server/api up and running quickly by defining a nice swagger doc that 
would transform into a code skeleton. imo, this would rock. it wouldn't 
be a one-stop-shop solution, but a nice solid toolkit to start from.


> There are a number of python integrations that swagger supports[3].
> Essentially, what these would allow us to do is use portions of the code
> to generate swagger specs. This means that our documentation would
> always be complete and accurate to what users actually experience when
> they interact with OpenStack APIs.

assuming we had the extra markup to make complete swagger docs it would 
always be up to date. i think the sticking point is getting to the level 
where the autogenerated output contains all the necessary information.

i think it would a great first step to have projects creating the 
baseline for their docs from an autogenerated format, then marking up as 
necessary. hopefully backporting the added information into their code, 
or just improving it to begin with.


> The automation of option tables in the configuration reference is a good
> parallel here. We used to have very incomplete and often outdated
> configuration tables. After the automation, literally hundreds of bugs
> have been solved with very little human intervention. The problem just
> doesn't exist anymore, and we've clawed back some of our precious resources.
>
> In the config-ref there are two types of auto-magic: non-swift, where
> all text lives in the code and updates should be done there; and swift,
> where the option name comes from the code, but the descriptive text must
> be manually entered - overwriting a default label of "No help text
> available for this option".
>
> Applying that to API documents, I think it would be amazing to try the
> former: it basically means that developers write their own API
> documentation, happily, in the code, in a format they are familiar with.
> We can take that and generate swagger for the API-ref and display it
> appropriately.

+1

i'd like to take it a step further and say that the projects themselves 
could be responsible for creating the swagger and then the upstream 
api-ref could grab those documents for aggregation into the doc team 
managed site. this will make more work for the project teams, but i like 
the idea of allowing the teams to have more control over their api output.

this might make it more difficult for the doc team to organize 
consistency as they might have to reach back to the project teams if 
there are issues with their generated docs, and i don't really have a 
good answer for that at this point. perhaps this could be an area where 
the team liaisons could help?


> However, that might not be possible, or might be too much work for a
> single release. So, the second option is still a very good one - we use
> automation to extract some information (eg it could be just class
> titles, doc strings, parameter names) to generate swagger specs with the
> placeholder text that we-as-the-docs-team can fill in with our best
> efforts - which are excellent, since we're the pros at strings :)

+1, this would be an awesome starting point


> The important thing is that this should be done continuously[4], as with
> the config-ref, to ensure accuracy and completeness. Just looking at the
> first 75 bugs in api-site[5], this approach would solve almost all of
> them. This isn't something you can say about simply switching formats ;)
>
> Anyway, thank you if you've managed to get to this point. I think we
> have a great opportunity here to make a very significant change to the
> way we do API documentation, and free up our previous docs resources to
> really focus on the things we do best, rather than spending a lot of
> time changing one arcane syntax into another :)

i agree, i find the prospect of this really encouraging =)

mike



> Regards,
>
>
>
> Tom
>
> [1] https://bugs.launchpad.net/openstack-api-site
> [2] https://review.openstack.org/#/c/177934/
> [3] https://github.com/swagger-api/swagger-spec#python , I found there's
> a worked example showing the output of the swagger UI for
> django-rest-swagger at:
> http://django-rest-swagger.readthedocs.org/en/latest/examples.html
> [4] Kinda once a milestone is how we're doing it with config-ref at the
> moment, but this could in theory be fully automated
> [5] https://bugs.launchpad.net/openstack-api-site/+bugs?orderby=id&start=0
>
> -------- Forwarded Message --------
> Subject: [Blueprint autogenerate-api-reference] Autogenerate API Reference
> Date: Mon, 27 Apr 2015 15:14:41 -0000
> From: Anne Gentle <anne at openstack.org>
> Reply-To: Anne Gentle <anne at openstack.org>
> To: tom at openstack.org
>
> Blueprint changed by Anne Gentle:
>
> Whiteboard set to:
> We don't really want autogenerated API documentation, because then how
> do you know if the code is correct? How can a quality engineer or SDK
> dev trust the output? Autogeneration is fine for the first set of docs
> but after that need to be treated as the "contract" for SDK developers
> to trust.
>
> --
> Autogenerate API Reference
> https://blueprints.launchpad.net/openstack-manuals/+spec/autogenerate-api-reference
>