Open Stack

Hi,

There has recently been some speculation around the V3 API and whether
we should go forward with it or instead backport many of the changes
to the V2 API. I believe that the core of the concern is the extra
maintenance and test burden that supporting two APIs means and the
length of time before we are able to deprecate the V2 API and return
to maintaining only one (well two including EC2) API again.

This email is rather long so here's the TL;DR version:

- We want to make backwards incompatible changes to the API
  and whether we do it in-place with V2 or by releasing V3
  we'll have some form of dual API support burden.
  - Not making backwards incompatible changes means:
    - retaining an inconsistent API
    - not being able to fix numerous input validation issues
    - have to forever proxy for glance/cinder/neutron with all
      the problems that entails.
  - Backporting V3 infrastructure changes to V2 would be a
    considerable amount of programmer/review time

- The V3 API as-is has:
  - lower maintenance
  - is easier to understand and use (consistent).
  - Much better input validation which is baked-in (json-schema)
    rather than ad-hoc and incomplete.

- Whilst we have existing users of the API we also have a lot more
  users in the future. It would be much better to allow them to use
  the API we want to get to as soon as possible, rather than trying
  to evolve the V2 API and forcing them along the transition that they
  could otherwise avoid.

- We already have feature parity for the V3 API (nova-network being
  the exception due to the very recent unfreezing of it), novaclient
  support, and a reasonable transition path for V2 users.

- Proposed way forward:
  - Release the V3 API in Juno with nova-network and tasks support
  - Feature freeze the V2 API when the V3 API is released
    - Set the timeline for deprecation of V2 so users have a lot
      of warning
    - Fallback for those who really don't want to move after
  deprecation is an API service which translates between V2 and V3
  requests, but removes the dual API support burden from Nova.

End TL;DR.


Although its late in the development cycle I think its important to
discuss this now rather than wait until the summit because if we go
ahead with the V3 API there is exploratory work around nova-network
that we would like to do before summit and it also affects how we look
at significant effort applying changes to the V2 API now. I'd also
prefer to hit the ground running with work we know we need to do in Juno
as soon as it opens rather than wait until the summit has finished.

Firstly I'd like to step back a bit and ask the question whether we
ever want to fix up the various problems with the V2 API that involve
backwards incompatible changes. These range from inconsistent naming
through the urls and data expected and returned, to poor and
inconsistent input validation and removal of all the proxying Nova
does to cinder, glance and neutron. I believe the answer to this is
yes - inconsistencies in the API make it harder to use (eg do I have a
instance or a server, and a project or a tenant just to name a
couple) and more error prone and proxying has caused several painful to
fix issues for us.

So at some point existing users of the API will need to make changes
or we'll effectively have to carry two APIs, whether it be inside the
V2 API code or split between a V2 implementation and V3
implementation. I think the number of changes required often makes it
easier to maintain such a split in different files rather than a
single one to avoid large slabs of if/else which makes the underlying
code harder to understand and more error prone in the eventual
removal. Its also more difficult to use decorators like we have for
input validation in V3 if the function also has to support the old
behaviour.

One approach that has been suggested for retaining the V2 API is to
gradually over time mark individual interfaces deprecated, support the
new behaviour in parallel and then after the deprecation period remove
the original behaviour.

With this I think we have to consider that if Open Stack continues to
be successful then although we already have an existing user base of
the V2 API, that with every release we have more and more new users
coming in. In fact in a few cycles we may have more post-Icehouse
users than pre-Icehouse ones. But by taking this gradual approach we
are basically saying to new users of the API that although we know
where we are headed with the API that they can't actually write
against it yet and instead have to use the V2 API.  And then every
cycle they will need to update their apps as we slowly deprecate parts
and replace them with the new interface. This seems to be a rather
hostile approach to people considering using our API. Also every
release we delay releasing the V3 API or defer making backwards
incompatible changes to V2 (if that's the route we take), the more
users we put into this situation of having to rework the software they
use to continue to access our API in the future.

Another side effect of this is new features such as tasks (which was
one of the significant reasons for not releasing the V3 API in
Icehouse) would not be able to be designed the way we want it in the
V2 API because it requires changes to the core API. So at least in the
short term we'd end up with a suboptimal API for tasks. And then go
through the pain of moving to the API we really want.

One thing to note is that the transition from V2 to V3 for users of
the API it does not have to be a big-bang thing. For example, an
application can quite legitimately create a server using the V2 API,
attach a volume using the V3 API, detach it using the V2 API and
delete the server using the V3 API. It is just a fairly thin layer on
top of the rest of Nova. So existing users of the API can decide
whether they want to tackle the job of moving from the V2 to V3 API in
one big step or in smaller ones over time.

We have also done a considerable amount of work in the V3 API in terms
of infrastructure which is not always visible to the API user. Such as
an improved plugin structure, better isolation between plugins,
versioning, better error handling, better input validation etc which
would all have to be backported to the V2 API. All in all I think if
you compare V2 and V3 API code, the latter is a lot cleaner and easier
to maintain. This is a non trivial amount of work that would take both
a lot of programmer and reviewer time in Juno, and perhaps overflow
into Koala. Time that has already been spent on the V3 API.

What I think our plan for the API transition should be is:

- Release the V3 API in Juno (it is probably too late to release it in
  Icehouse at this stage now without a bunch of FFEs, but
  theoretically we could make the skeleton of the tasks API changes in
  IceHouse without functional tasks which would allow tasks to be
  added in a backwards compatible manner in Juno). The Juno release
  would have both full task and nova-network support.

- Feature freeze the V2 API development in Juno or at the very least
  when the V3 API is marked as current/supported (bug fixes are of
  course ok). I have several reasons for wanting to do this. The first
  is to avoid the burden of actively developing two APIs. We already
  have feature parity in V3 with V2 - with the exception of
  nova-network which was deliberately left out of the V3 API because
  it was considered deprecated, but has recently been re-opened for
  development. So an exception for V2 API nova network development
  would seem reasonable until the V3 API supports it fully.

  I think it's a pretty unusual situation where a project decides to
  continue significant feature development on both the latest version
  and the next most recent version. We don't for example allow feature
  development in Havana. And I don't think the situation needs to be
  any different for V2 once V3 is available. We already have feature
  parity and we have a reasonable transition plan for existing users
  so I think its quite reasonable to say to users that if they want
  new features that they need to access them via the new API.

  Although as mentioned above, its possible to use the V3 API for new
  features even if a user is using the V2 API for everything else. So
  any new features could still be accessed by people who just to only
  modify their existing V2 API based programs to take advantage of the
  new functionality without modifying the rest. Also deployers, if they
  wished, could through policy only expose the parts of the V3 API that
  they want (even though the core would have to be loaded it doesn't
  necessarily have to be accessible to everyone).

- At summit or the Juno midcycle meetup decide on a release when we will
  remove the V2 API so existing users of the V2 API have plenty of
  warning. They can decide either to do a gradual transition or a
  big-bang move to the V3 API. Ultimately where we set the date for
  the removal of the V2 API is going to be a balance between needs of
  users and what we can afford to spend on maintenance of it. But
  whether we try to evolve the V2 API or move to the V3 API we still
  need to make that decision. If we choose the V3 API we can also
  still choose to gradually deprecate the V2 over time (eg remove
  support for rarely used extensions earlier and point users to the V3
  API for that functionality).

- For those who *really* *really* don't want to move off the V2 API
  when the V2 API is removed, there is the possibility of writing a
  separate translation service which takes V2 API REST requests and
  translates them to V3 API requests and also does the proxying that
  the V2 API currently does for glance/neutron/cinder. Its not a
  trivial job, but should be possible and does remove the burden of
  supporting two APIs in the Nova tree.


In summary, we need to fix the problems with the V2 API which requires
changes which are backwards incompatible. I believe releasing the V3
API in Juno and setting a deprecation timeframe for V2 well in advance
is the best overall solution we have for both our users (both current
and future) and us as developers - it balances the needs of users of
our API and what the easiest path for us as developers is. Attempting
to "evolve" the V2 API is on the surface attractive because we don't
in the short term need to make any decisions around deprecation but
will in the longer term involve more pain for everyone.

Chris