Open Stack

Sorry for the top post. I was asked to look at this thread and toss in 
my €0.02 but I do not believe I could possibly read this whole thread - 
I'm too jet lagged. So...

As a current large-scale production consumer of the API, I don't care at 
all as long as python-novaclient keeps working at the Python Library API 
level. I do  not care about the REST api specifics at all. I would 
imagine that most Java folks only care that jclouds works. I would 
imagine that most Ruby users would only care that fog works. Since 
jclouds already supports a billion apis, I doubt supporting v2 and v3 
would be hard for them.

AND

As long as the v2/v3 switch is discoverable by the library and I don't, 
as a consumer of the library, need to know about it - so that 
python-novaclient will continue to support client operations on both 
REST versions - I'm fine with that - because I want to continue to be 
able to operate the old Diablo-based HP cloud, the new trunk HP cloud, 
the trunk Rackspace cloud and the TripleO cloud with the same scripts.

That's what I want. I'm sure other people want other things.

On 02/24/2014 07:50 AM, Christopher Yeoh wrote:
> Hi,
>
> There has recently been some speculation around the V3 API and whether
> we should go forward with it or instead backport many of the changes
> to the V2 API. I believe that the core of the concern is the extra
> maintenance and test burden that supporting two APIs means and the
> length of time before we are able to deprecate the V2 API and return
> to maintaining only one (well two including EC2) API again.
>
> This email is rather long so here's the TL;DR version:
>
> - We want to make backwards incompatible changes to the API
>    and whether we do it in-place with V2 or by releasing V3
>    we'll have some form of dual API support burden.
>    - Not making backwards incompatible changes means:
>      - retaining an inconsistent API
>      - not being able to fix numerous input validation issues
>      - have to forever proxy for glance/cinder/neutron with all
>        the problems that entails.
>    - Backporting V3 infrastructure changes to V2 would be a
>      considerable amount of programmer/review time
>
> - The V3 API as-is has:
>    - lower maintenance
>    - is easier to understand and use (consistent).
>    - Much better input validation which is baked-in (json-schema)
>      rather than ad-hoc and incomplete.
>
> - Whilst we have existing users of the API we also have a lot more
>    users in the future. It would be much better to allow them to use
>    the API we want to get to as soon as possible, rather than trying
>    to evolve the V2 API and forcing them along the transition that they
>    could otherwise avoid.
>
> - We already have feature parity for the V3 API (nova-network being
>    the exception due to the very recent unfreezing of it), novaclient
>    support, and a reasonable transition path for V2 users.
>
> - Proposed way forward:
>    - Release the V3 API in Juno with nova-network and tasks support
>    - Feature freeze the V2 API when the V3 API is released
>      - Set the timeline for deprecation of V2 so users have a lot
>        of warning
>      - Fallback for those who really don't want to move after
>    deprecation is an API service which translates between V2 and V3
>    requests, but removes the dual API support burden from Nova.
>
> End TL;DR.
>
>
> Although its late in the development cycle I think its important to
> discuss this now rather than wait until the summit because if we go
> ahead with the V3 API there is exploratory work around nova-network
> that we would like to do before summit and it also affects how we look
> at significant effort applying changes to the V2 API now. I'd also
> prefer to hit the ground running with work we know we need to do in Juno
> as soon as it opens rather than wait until the summit has finished.
>
> Firstly I'd like to step back a bit and ask the question whether we
> ever want to fix up the various problems with the V2 API that involve
> backwards incompatible changes. These range from inconsistent naming
> through the urls and data expected and returned, to poor and
> inconsistent input validation and removal of all the proxying Nova
> does to cinder, glance and neutron. I believe the answer to this is
> yes - inconsistencies in the API make it harder to use (eg do I have a
> instance or a server, and a project or a tenant just to name a
> couple) and more error prone and proxying has caused several painful to
> fix issues for us.
>
> So at some point existing users of the API will need to make changes
> or we'll effectively have to carry two APIs, whether it be inside the
> V2 API code or split between a V2 implementation and V3
> implementation. I think the number of changes required often makes it
> easier to maintain such a split in different files rather than a
> single one to avoid large slabs of if/else which makes the underlying
> code harder to understand and more error prone in the eventual
> removal. Its also more difficult to use decorators like we have for
> input validation in V3 if the function also has to support the old
> behaviour.
>
> One approach that has been suggested for retaining the V2 API is to
> gradually over time mark individual interfaces deprecated, support the
> new behaviour in parallel and then after the deprecation period remove
> the original behaviour.
>
> With this I think we have to consider that if Open Stack continues to
> be successful then although we already have an existing user base of
> the V2 API, that with every release we have more and more new users
> coming in. In fact in a few cycles we may have more post-Icehouse
> users than pre-Icehouse ones. But by taking this gradual approach we
> are basically saying to new users of the API that although we know
> where we are headed with the API that they can't actually write
> against it yet and instead have to use the V2 API.  And then every
> cycle they will need to update their apps as we slowly deprecate parts
> and replace them with the new interface. This seems to be a rather
> hostile approach to people considering using our API. Also every
> release we delay releasing the V3 API or defer making backwards
> incompatible changes to V2 (if that's the route we take), the more
> users we put into this situation of having to rework the software they
> use to continue to access our API in the future.
>
> Another side effect of this is new features such as tasks (which was
> one of the significant reasons for not releasing the V3 API in
> Icehouse) would not be able to be designed the way we want it in the
> V2 API because it requires changes to the core API. So at least in the
> short term we'd end up with a suboptimal API for tasks. And then go
> through the pain of moving to the API we really want.
>
> One thing to note is that the transition from V2 to V3 for users of
> the API it does not have to be a big-bang thing. For example, an
> application can quite legitimately create a server using the V2 API,
> attach a volume using the V3 API, detach it using the V2 API and
> delete the server using the V3 API. It is just a fairly thin layer on
> top of the rest of Nova. So existing users of the API can decide
> whether they want to tackle the job of moving from the V2 to V3 API in
> one big step or in smaller ones over time.
>
> We have also done a considerable amount of work in the V3 API in terms
> of infrastructure which is not always visible to the API user. Such as
> an improved plugin structure, better isolation between plugins,
> versioning, better error handling, better input validation etc which
> would all have to be backported to the V2 API. All in all I think if
> you compare V2 and V3 API code, the latter is a lot cleaner and easier
> to maintain. This is a non trivial amount of work that would take both
> a lot of programmer and reviewer time in Juno, and perhaps overflow
> into Koala. Time that has already been spent on the V3 API.
>
> What I think our plan for the API transition should be is:
>
> - Release the V3 API in Juno (it is probably too late to release it in
>    Icehouse at this stage now without a bunch of FFEs, but
>    theoretically we could make the skeleton of the tasks API changes in
>    IceHouse without functional tasks which would allow tasks to be
>    added in a backwards compatible manner in Juno). The Juno release
>    would have both full task and nova-network support.
>
> - Feature freeze the V2 API development in Juno or at the very least
>    when the V3 API is marked as current/supported (bug fixes are of
>    course ok). I have several reasons for wanting to do this. The first
>    is to avoid the burden of actively developing two APIs. We already
>    have feature parity in V3 with V2 - with the exception of
>    nova-network which was deliberately left out of the V3 API because
>    it was considered deprecated, but has recently been re-opened for
>    development. So an exception for V2 API nova network development
>    would seem reasonable until the V3 API supports it fully.
>
>    I think it's a pretty unusual situation where a project decides to
>    continue significant feature development on both the latest version
>    and the next most recent version. We don't for example allow feature
>    development in Havana. And I don't think the situation needs to be
>    any different for V2 once V3 is available. We already have feature
>    parity and we have a reasonable transition plan for existing users
>    so I think its quite reasonable to say to users that if they want
>    new features that they need to access them via the new API.
>
>    Although as mentioned above, its possible to use the V3 API for new
>    features even if a user is using the V2 API for everything else. So
>    any new features could still be accessed by people who just to only
>    modify their existing V2 API based programs to take advantage of the
>    new functionality without modifying the rest. Also deployers, if they
>    wished, could through policy only expose the parts of the V3 API that
>    they want (even though the core would have to be loaded it doesn't
>    necessarily have to be accessible to everyone).
>
> - At summit or the Juno midcycle meetup decide on a release when we will
>    remove the V2 API so existing users of the V2 API have plenty of
>    warning. They can decide either to do a gradual transition or a
>    big-bang move to the V3 API. Ultimately where we set the date for
>    the removal of the V2 API is going to be a balance between needs of
>    users and what we can afford to spend on maintenance of it. But
>    whether we try to evolve the V2 API or move to the V3 API we still
>    need to make that decision. If we choose the V3 API we can also
>    still choose to gradually deprecate the V2 over time (eg remove
>    support for rarely used extensions earlier and point users to the V3
>    API for that functionality).
>
> - For those who *really* *really* don't want to move off the V2 API
>    when the V2 API is removed, there is the possibility of writing a
>    separate translation service which takes V2 API REST requests and
>    translates them to V3 API requests and also does the proxying that
>    the V2 API currently does for glance/neutron/cinder. Its not a
>    trivial job, but should be possible and does remove the burden of
>    supporting two APIs in the Nova tree.
>
>
> In summary, we need to fix the problems with the V2 API which requires
> changes which are backwards incompatible. I believe releasing the V3
> API in Juno and setting a deprecation timeframe for V2 well in advance
> is the best overall solution we have for both our users (both current
> and future) and us as developers - it balances the needs of users of
> our API and what the easiest path for us as developers is. Attempting
> to "evolve" the V2 API is on the surface attractive because we don't
> in the short term need to make any decisions around deprecation but
> will in the longer term involve more pain for everyone.
>
> Chris
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>