Open Stack

There has been quite a bit of discussion about the future of the v3 API
recently.  There has been growing support for the idea that we should
change course and focus on evolving the existing v2 API instead of
putting out a new major revision.  This message is a more complete
presentation of that proposal that concludes that we can do what we
really need to do with only the v2 API.

Keeping only the v2 API requires some confidence that we can stick with
it for years to come.  We don't want to be revisiting this any time
soon.  This message addresses a bunch of different questions about how
things would work if we only had v2.

1) What about tasks?

In some cases, the proposed integration of tasks is backwards
compatible.  A task ID will be added to a header.  The biggest point of
debate was if and how we would change the response for creating a
server.  For tasks in v2, we would not change the response by default.
The task ID would just be in a header.  However, if and when the client
starts exposing version support information, we can provide an
alternative/preferred response based on tasks.

For example:

   Accept: application/json;type=task

2) Versioning extensions

One of the points being addressed in the v3 API was the ability to
version extensions.  In v2, we have historically required new API
extensions, even for changes that are backwards compatible.  We propose
the following:

 - Add a version number to v2 API extensions
 - Allow backwards compatible changes to these API extensions,
accompanied by a version number increase
 - Add the option to advertise an extension as deprecated, which can be
used for all those extensions created only to advertise the availability
of new input parameters

3) Core versioning

Another pain point in API maintenance has been having to create API
extensions for every small addition to the core API.  We propose that a
version number be exposed for the core API that exposes the revision of
the core API in use.  With that in place, backwards compatible changes
such as adding a new property to a resource would be allowed when
accompanied by a version number increase.

With versioning of the core and API extensions, we will be able to cut
down significantly on the number of changes that require an API
extension without sacrificing the ability of a client to discover
whether the addition is present or not.

4) API Proxying

We don't see proxying APIs as a problem.  It is the cost we pay for
choosing to split apart projects after they are released.  We don't
think it's fair to break users just because we have chosen to split
apart the backend implementation.

Further, the APIs that are proxied are frozen while those in the other
projects are evolving.  We believe that as more features are available
only via the native APIs in Cinder, Glance, and Neutron, users will
naturally migrate over to the native APIs.

Over time, we can ensure clients are able to query the API without the
need to proxy by adding new formats or extensions that don't return data
that needed to be proxied.

5) Capitalization and Naming Consistency

Some of the changes in the v3 API included changes to capitalization and
naming for improved consistency.  If we stick with v2 only, we will not
be able to make any of these changes.  However, we believe that not
breaking any existing clients and not having to maintain a second API is
worth not making these changes, or supporting some indirection to
achieve this for newer clients if we decide it is important.

6) Response Code Consistency and Correctness

The v2 API has many places where the response code returned for a given
operation is not strictly correct. For example a 200 is returned when a
202 would be more appropriate. Correcting these issues should be
considered for improving the future use of the API, however there does
not seem to be any support for considering this a critical problem right
now. There are two approaches that can be taken to improve this in v2:

Just fix them. Right now, we return some codes that imply we have dealt
with a request, when all we have done is queue it for processing (and
vice versa). In the future, we may change the backend in such a way that
a return code needs to change to continue to be accurate anyway. If we
just assume that return codes may change to properly reflect the action
that was taken, then we can correct existing errors and move on.
Accept them as wrong but not critically so. With this approach, we can
strive for correctness in the future without changing behavior of our
existing APIs. Nobody seems to complain about them right now, so
changing them seems to be little gain. If the client begins exposing a
version header (which we need for other things) then we could
alternately start returning accurate codes for those clients.

The key point here is that we see a way forward with this in the v2 API
regardless of which path we choose.

7) Entrypoint based extensions

The v3 effort included improvements to the infrastructure used to
implement the API, both for proper extensions and modular construction
of the core API.  We definitely want that for v2, and since these are
just internal implementation details, there is no reason why we can't
implement these improvements in the v2 API.  Note that with the addition
of versioning of extensions and the core, we should be adding fewer API
extensions and may be able to collapse some that we already have.

8) Input Validation

Previously, much of our input validation happened at the database driver
layer for various reasons. This means that behavior of the API depends
on such user-invisible things as which RDBM is in use. Thus, our input
validation is already inconsistent across deployments. Further, the move
to objects as the communication mechanism between the API and the
backend means that more input validation is going on than we once had
anyway, and this is not something we can avoid unless we freeze the
backend anyway.

Exposing a definition like jsonschema is something that we should do
anyway, and the process of doing so should allow us to define what was
previously undefined. Given the variance of the behavior depending on
the database used on the backend, getting something reasonably strict
should be roughly equivalent (user-wise) to working against a provider
that was using something other than MySQL.

9) v3 ... if not now, when?

Not in the foreseeable future.  We don't see it happening unless some
group of people wanted to rebuild the API from scratch in such a way
that it doesn't look anything like the current API.  The new API would
have to be different and compelling enough that we'd be willing to
maintain it along side the current API for several years.

10) Conclusion and Proposal

The v3 API effort has produced a lot of excellent work.  However, the
majority opinion seems to be that we should avoid the cost of
maintaining two APIs if at all possible.  We should apply what has been
learned to the existing API where we can and focus on making v2
something that we can continue to maintain for years to come.

We recognize and accept that it is a failure of Nova project leadership
that we did not come to this conclusion much sooner.  We hope to have
learned from the experience to help avoiding a situation like this
happening again in the future.

Please provide your input on this proposal, even if it is just agreement
with this as the way forward.

Thanks,

Proposal authors / sponsors:

    Russell Bryant
    Dan Smith
    John Garbutt
    Andrew Laski