Open Stack

On Thu, 20 Feb 2014 08:22:57 -0500
Sean Dague <sean at dague.net> wrote:
> 
> We're also duplicating a lot of test and review energy in having 2 API
> stacks. Even before v3 has come out of experimental it's consumed a
> huge amount of review resource on both the Nova and Tempest sides to
> get it to it's current state.
> 
> So my feeling is that in order to get more energy and focus on the
> API, we need some kind of game plan to get us to a single API
> version, with a single data payload in L (or on the outside, M). If
> the decision is v2 must be in both those releases (and possibly
> beyond), then it seems like asking other hard questions.
> 
> * why do a v3 at all? instead do we figure out a way to be able to
> evolve v2 in a backwards compatible way.

So there's lots of changes (cleanups) made between v2 and v3 which are
really not possible to do in a backwards compatible way. One example
is that we're a lot stricter and consistent on input validation in v3
than v2 which is better both from a user and server point of view.
Another is that the tasks API would be a lot uglier and really look
"bolted on" if we tried to do so. Also doing so doesn't actually reduce
the test load as if we're still supporting the old 'look' of the api we
still need to test for it separately to the new 'look' even if we don't
bump the api major version. 

In terms of code sharing (and we've experimented a bit with this for
v2/v3) I think in most cases ends up actually being easier having two
quite completely separate trees because it ends up diverging so much
that having if statements around everywhere to handle the different
cases is actually a higher maintenance burden (much harder to read)
than just knowing that you might have to make changes in two quite
separate places. 

> * if we aren't doing a v3, can we deprecate XML in v2 in Icehouse so
> that working around all that code isn't a velocity inhibitor in the
> cleanups required in v2? Because some of the crazy hacks that exist to
> make XML structures work for the json in v2 is kind of special.

So I don't think we can do that for similar reasons we can't just drop
V2 after a couple of cycles. We should be encouraging people off, not
forcing them off. 

> This big bang approach to API development may just have run it's
> course, and no longer be a useful development model. Which is good to
> find out. Would have been nice to find out earlier... but not all
> lessons are easy or cheap. :)

So I think what the v3 gives us is much more consistent and clean
API base to start from. It's a clean break from the past. But we have to
be much more careful about any future API changes/enhancements than we
traditionally have done in the past especially with any changes which
affect the core. I think we've already significantly raised the quality
bar in what we allow for both v2 and v3 in Icehouse compared to previous
releases (those frustrated with trying to get API changes in will
probably agree) but I'd like us to get even stricter about it in the
future because getting it wrong in the API design has a MUCH higher
long term impact than bugs in most other areas. Requiring an API spec
upfront (and reviewing it) with a blueprint for any new API features
should IMO be compulsory before a blueprint is approved. 

Also micro and extension versioning is not the magic bullet which will
get us out of trouble in the future. Especially with the core changes.
Because even though versioning allows us to make changes, for similar
reasons to not being able to just drop V2 after a couple of cycles
we'll still need to keep supporting (and testing) the old behaviour for
a significant period of time (we have often quietly ignored
this issue in the past).

Ultimately the only way to free ourselves from the maintenance of two
API versions (and I'll claim this is rather misleading as it actually
has more dimensions to it than this) is to convince users to move from
the V2 API to the "new one". And it doesn't make much difference
whether we call it V3 or V2.1 we still have very similar maintenance
burdens if we want to make the sorts of API changes that we have done
for V3.

Chris

> 
> 	-Sean
> 
> On 02/19/2014 12:36 PM, Russell Bryant wrote:
> > Greetings,
> > 
> > The v3 API effort has been going for a few release cycles now.  As
> > we approach the Icehouse release, we are faced with the following
> > question: "Is it time to mark v3 stable?"
> > 
> > My opinion is that I think we need to leave v3 marked as
> > experimental for Icehouse.
> > 
> > There are a number of reasons for this:
> > 
> > 1) Discussions about the v2 and v3 APIs at the in-person Nova meetup
> > last week made me come to the realization that v2 won't be going
> > away *any* time soon.  In some cases, users have long term API
> > support expectations (perhaps based on experience with EC2).  In
> > the best case, we have to get all of the SDKs updated to the new
> > API, and then get to the point where everyone is using a new enough
> > version of all of these SDKs to use the new API.  I don't think
> > that's going to be quick.
> > 
> > We really don't want to be in a situation where we're having to
> > force any sort of migration to a new API.  The new API should be
> > compelling enough that everyone *wants* to migrate to it.  If
> > that's not the case, we haven't done our job.
> > 
> > 2) There's actually quite a bit still left on the existing v3 todo
> > list. We have some notes here:
> > 
> > https://etherpad.openstack.org/p/NovaV3APIDoneCriteria
> > 
> > One thing is nova-network support.  Since nova-network is still not
> > deprecated, we certainly can't deprecate the v2 API without
> > nova-network support in v3.  We removed it from v3 assuming
> > nova-network would be deprecated in time.
> > 
> > Another issue is that we discussed the tasks API as the big new API
> > feature we would include in v3.  Unfortunately, it's not going to be
> > complete for Icehouse.  It's possible we may have some initial parts
> > merged, but it's much smaller scope than what we originally
> > envisioned. Without this, I honestly worry that there's not quite
> > enough compelling functionality yet to encourage a lot of people to
> > migrate.
> > 
> > 3) v3 has taken a lot more time and a lot more effort than anyone
> > thought.  This makes it even more important that we're not going to
> > need a v4 any time soon.  Due to various things still not quite
> > wrapped up, I'm just not confident enough that what we have is
> > something we all feel is Nova's API of the future.
> > 
> > 
> > Let's all take some time to reflect on what has happened with v3 so
> > far and what it means for how we should move forward.  We can
> > regroup for Juno.
> > 
> > Finally, I would like to thank everyone who has helped with the
> > effort so far.  Many hours have been put in to code and reviews for
> > this.  I would like to specifically thank Christopher Yeoh for his
> > work here. Chris has done an *enormous* amount of work on this and
> > deserves credit for it.  He has taken on a task much bigger than
> > anyone anticipated. Thanks, Chris!
> > 
> 
>