Open Stack

Hey

Almost exactly 6 months ago, the PPB had a discussion about 3rd party
APIs after a mailing list discussion on the topic:

  http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-08-20.00.html
  https://lists.launchpad.net/openstack-poc/msg00448.html

The conclusion reached in the meeting was:

  "an OpenStack project will support an official API in it's core 
   implementation (the OpenStack API). other APIs will be implemented
   external to core. the core project will expose stable, complete,
   performant interfaces so that 3rd party APIs can be implemented in a
   complete and performant manner."

Let me first qualify that with some of the discussion that lead up to
the policy being voted on:

  <vishy> jbryce: I'm not sure that we can mandate that they must live outside
  <vishy> jbryce: unless we are just talking ultimate goal here.
  <vishy> jbryce: I agree with the principle that they should be there
  <vishy> jbryce: I just don't know if we are technically at the point where they can
  <jbryce> vishy: let's work on defining the end state of this as the goal/convention to aim for

i.e. I think it's fair to say the "policy" was intended to be a
forward-looking "where we want to get to" statement.

Also, the statement might seem to imply that 3rd party APIs should be
implemented as proxies to the OpenStack REST APIs, but actually the
statement wasn't directing an implementation choice. The other option
discussed was a project exposing a "stable, complete, performant" python
interface for implementing 3rd party APIs either as a plugins or
completely separate services talking to the DB and message bus.


Now, what about Nova?

Nova has an implementation of the EC2 API (a "3rd party API") and,
indeed, has had this since the project began ... even before the
Rackspace CloudServers API became the OpenStack API.

It seems to me that the EC2 API is a fairly popular feature, generally
works reasonably well and is tested by unit tests, the devstack gate
(using devstack's euca exercise) and smokestack (using nova smoketests).

It's true there isn't any particular person or group of people
consistently working to improve the EC2 API (can that also be said of
the OpenStack API?) but in Folsom nova/api/ec2 did have 59 commits (4%
of total) by 27 developers (14% of total) and the diffstat was:

   6 files changed, 585 insertions(+), 688 deletions(-)

Part of that work involved re-factoring some code that was common
between the EC2 and OpenStack APIs.

Anyway, my take is the EC2 API is a nice feature, we should be proud of
it and we're successfully maintaining it without a terrible amount of
pain.


In theory, the PPB's policy decision should have prompted some move by
Nova developers to figure out what "stable, complete, performant"
interface to expose and move the EC2 API to that. Six months have passed
with zero progress, partly because of a lack of interest but mostly
because Nova developers have a lot of other higher priority things on
their plate.


The question reared its head again at the Grizzly summit when Joe Gordon
lead this session:

  https://etherpad.openstack.org/grizzly-nova-splitting-out-EC2

The (or at least part of the) motivation for the session was Joe and his
team's work on a Google Compute Engine API implementation:

  https://github.com/cloudscaling/nova-gce

This is implemented quite similarly to the EC2 API so the question
becomes "if new 3rd party APIs can't be added to Nova, how do we expose
the appropriate APIs for those new APIs". It was pretty clear during the
session that we don't have an answer to that.


So, to the point - what Nova developers wish to discuss here and at a TC
meeting is how we can make progress.

Assuming we don't have the "stable, complete, performant" interface
needed for 3rd party APIs, what do we do if the GCE API is proposed for
inclusion in Nova?


My opinion - the GCE work looks awesome, it's an exciting new feature
and it's the kind of awesome new feature by awesome developers that we
should be facilitating.

We have a long term goal of facilitating work like this happening
outside of Nova, but no real sign of that goal coming about in the
medium term.

We shouldn't let our idealistic long term plans get in the way of
pragmatic, exciting progress happening now.

Cheers,
Mark.