[openstack-dev] [docs] [api] Swagger limitations?

Jamie Hannaford jamie.hannaford at rackspace.com
Tue May 17 14:13:51 UTC 2016

All great points, thanks Anne and Sean. My responses are inline.

From: Sean Dague <sean at dague.net>
Sent: 17 May 2016 15:30
To: openstack-dev at lists.openstack.org
Subject: Re: [openstack-dev] [docs] [api] Swagger limitations?

On 05/17/2016 08:57 AM, Jamie Hannaford wrote:
> Hi all,
> Back in March Anne wrote a great summary on the current state of Swagger
> for OpenStack
> docs: http://lists.openstack.org/pipermail/openstack-dev/2016-March/090659.html.
> <http://lists.openstack.org/pipermail/openstack-dev/2016-March/090659.html>
>  Is this still the current state of things? Magnum is looking to
> document its API soon, so I want to look at the feasibility of doing it
> in Swagger. If it's not possible, RST should suffice.
> In terms of the specific limitations that were listed:
> - the /actions endpoint. Would it be possible to use vendor extensions
> to solve this problem? For example, imagine that you want to document
> reboot and rebuild operations. You could create custom verb properties
> for these operations (`x-post-reboot`, `x-post-rebuild`) that would
> allow differentiation of the JSON payload under the same URL category.
> The HTML rendering would need to be adapted to take this into
> consideration. [1]

If you do this you're now outside the swagger spec, which means that all
the toolchains in swagger that exist to:

* build docs
* build clients
* build tests

No longer apply to your API.

So yes, you could do this, but in doing so you abandon most of the
ecosystem that makes swagger interesting, and now have to maintain all
your own dedicated tooling.

There is no swagger HTML rendering project in OpenStack that can be
simply modified for this. One of the advantages for swagger was that
would be an upstream ecosystem tool to build things like the API site.
You now give that up and take on a multi person year engineering effort
to build a custom documentation site rendering engine (probably in
javascript, as that's where most of the starting components are).
Swagger is also pretty non trivial in the way references are resolved,
so this may end up a *really* big engineering effort.

I agree, but I don't think it's an unreasonable compromise to make because, at least in the context of OpenStack's ecosystem, we already provide much of that tooling to end-users. We have a plethora of clients already out there and a project underway to unify them into a single client. So I don't think we'd be putting users at a disadvantage by using our own vendor extensions, so long as they're well designed and well documented.

The only inconvenience would be to adapt the build process to take custom verbs into consideration. In my opinion that's more of a short-term software challenge for contributors rather than an insurmountable one that affects end-users. Then again, I don't have much experience with the build/infra process, so please call me out if that's a complete mischaracterisation :)

While I accept that it's definitely preferable to use upstream tools when they're useful, with such a complex beast like OpenStack, how long do you think we could carry on using vanilla upstream tools? At some point we would probably need to roll out our own modifications to satisfy our use cases. In terms of it requiring a multi-person engineering effort, isn't that already the remit of the current docs team?

> - microversions. The more I think about it, the more I realise that this
> is not really a Swagger limitation but more of a consideration that
> *any* documentation format needs to solve. It seems that per
> microversion release, a separate doc artefact is required which
> encapsulates the API at that point in time. This would be very easy to
> do in Swagger compared to RST (single file vs directory of RST files).

You could go that route, and depending on how often you change your
format, and how big your API is, it may or may not be cumbersome. Fixes
to a base resource would require fixing it in N copies of the file.

If my understanding is correct, this would not happen. Each microversion, when released, is locked and would be preserved in that doc artefact. If a new software release is needed to fix a bug, that change would be encapsulated in a microversion with its own doc file.

It would mean that users of your API must preselect the API version
first before seeing anything. And they will only see a pristine document
at that version.

They will not have access to change notes inline (i.e. "New in 2.12").
When looking at consuming an API. IMHO, that is something which is
incredibly useful in reading an API. It's existence in the python
standard lib makes it much clearer about whether you want to take
advantage of feature X, or decide you'd rather have compatibility, and not.

I definitely agree that this information is useful for the user, but I think it belongs in a RELEASENOTES format, not API docs. I think we should get to a position where API versioning is treated with the importance it deserves. The user should understand what version they want to interact with and make a decision, and then be presented with that API documentation.

In terms of the logic of how we can re-use doc artefacts, we could do something like the following:

* if a microversion which is selected by a user has a specific doc artefact, show it
* if a specific microversion document does not exist, find its nearest sibling. For example, if no document was published for microversions 1.100 or 1.99, but there was one published for 1.98, it is reasonable to assume that both 1.100 and 1.99 have the same API as 1.98, and can therefore use its doc artefact

As with everything, it's trade offs, and what you want to get out of it.

Swagger is a really interesting API Design Language. And if you start
with Swagger when designing your API you impose a set of constraints
that give you a bunch of benefits. If you go beyond it's constraints,
you have imposed a very large and novel DSL on your environment (which
is strictly no longer swagger), which means you get all the costs of
conforming to an external standard without any of the benefits.

I think that our actions interface turned out to be a mistake, but it's
one made so long ago that it's pretty embedded. Long term getting
ourselves out of that hole would be good.

Completely agree. The good news is that the API-WG have a spec in review (https://review.openstack.org/#/c/234994/) which might solve these problems. They want to make actions first-class resources in the API. From my initial understanding, this could be represented in Swagger without the need for vendor extensions.

The microversions model is something that I think would be worth
engaging the OpenAPI upstream community. Our current implementation is
pretty OpenStack specific, but the concept would apply to any open
source project with a REST API that is CDed at different rates by
different deployers.


Sean Dague

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe

Rackspace International GmbH a company registered in the Canton of Zurich, Switzerland (company identification number CH- whose registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland. Rackspace International GmbH privacy policy can be viewed at www.rackspace.co.uk/legal/swiss-privacy-policy - This e-mail message may contain confidential or privileged information intended for the recipient. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at abuse at rackspace.com and delete the original message. Your cooperation is appreciated.

More information about the OpenStack-dev mailing list