Open Stack

On Mon, 24 Feb 2014 07:56:19 -0800
Dan Smith <dms at danplanet.com> wrote:

> > - We want to make backwards incompatible changes to the API
> >   and whether we do it in-place with V2 or by releasing V3
> >   we'll have some form of dual API support burden.
> 
> IMHO, the cost of maintaining both APIs (which are largely duplicated)
> for almost any amount of time outweighs the cost of localized changes.

The API layer is a actually quite a very thin layer on top of the rest
of Nova. Most of the logic in the API code is really just checking
incoming data, calling the underlying nova logic and then massaging
what is returned in the correct format. So as soon as you change the
format the cost of localised changes is pretty much the same as
duplicating the APIs. In fact I'd argue in many cases its more because
in terms of code readability its a lot worse and techniques like using
decorators for jsonschema for input validation are a lot harder to
implement. And unit and tempest tests still need to be duplicated.

> 
> The neutron stickiness aside, I don't see a problem leaving the
> proxying in place for the foreseeable future. I think that it's
> reasonable to mark them as deprecated, encourage people not to use
> them, and maybe even (with a core api version to mark the change) say
> that they're not supported anymore.
> 

I don't understand why this is also not seen as forcing people off V2
to V3 which is being given as a reason for not being able to set a
reasonable deprecation time for V2. This will require major changes for
people using the V2 API to change how they use it. 


> I also think that breaking our users because we decided to split A
> into B and C on the backend kind of sucks. I imagine that continuing
> to do that at the API layer (when we're clearly going to keep doing
> it on the backend) is going to earn us a bit of a reputation.

In all the discussions we've (as in the Nova group) had over the API
there has been a pretty clear consensus that proxying is quite
suboptimal (there are caching issues etc) and the long term goal is to
remove it from Nova. Why the change now? 

> 
> >   - Backporting V3 infrastructure changes to V2 would be a
> >     considerable amount of programmer/review time
> 
> While acknowledging that you (and others) have done that for v3
> already, I have to think that such an effort is much less costly than
> maintaining two complete overlapping pieces of API code.

I strongly disagree here. I think you're overestimating the
amount of maintenance effort this involves and significantly
underestimating how much effort and review time a backport is going to
take.

> - twice the code
> - different enough to be annoying to convert existing clients to use
> - not currently different enough to justify the pain

For starters, It's not twice the code because we don't do things like
proxying and because we are able to logically separate out input
validation jsonschema. 

v2 API: ~14600 LOC
v3 API: ~7300 LOC (~8600 LOC if nova-network as-is added back in,
though the actually increase would almost certainly be a lot smaller)

And that's with a lot of the jsonschema patches not landed. So its
actually getting *smaller*. Long term which looks the better from a
maintenance point of view 

And I think you're continuing to look at it solely from the point of
view of pain for existing users of the API and not considering the pain
for new users who have to work out how to use the API. Eg just one
simple example, but how many people new to the API get confused about
what they are meant to send when it asks for instance_uuid when
they've never received one - is at server uuid - and if so what's the
difference? Do I have to do some sort of conversion? Similar issues
around project and tenant. And when writing code they have to remember
for this part of the API they pass it as server_uuid, in another
instance_uuid, or maybe its just id? All of these looked at
individually may look like small costs or barriers to using the API but
they all add up and they end up being imposed over a lot of people.

> This feels a lot like holding our users hostage in order to get them
> to move. We're basically saying "We tweaked a few things, fixed some
> spelling errors, and changed some date stamp formats. You will have to
> port your client, or no new features for you!" That's obviously a
> little hyperbolic, but I think that deployers of APIv2 would probably
> feel like that's the story they have to give to their users.

And how is say removing proxying or making *any* backwards incompatible
change any different? And this sort of situation is very common with
major library version upgrades. If you want new features you have to
port to the library version which requires changes to your app (that's
why its a major library version not a minor one).

> I naively think that we could figure out a way to move things forward
> without having to completely break older clients. It's clear that
> other services (with much larger and more widely-used APIs) are able
> to do it.

Well if you never deprecate the only way to do it is to maintain the
old API forever (including test). And just take the hit on all that
involves.

> That said, I think the corollary to the above question is: do we ever
> want to knowingly break an existing client for either of:
> 
> 1. arbitrary user-invisible backend changes in implementation or
>    service organization
> 2. purely cosmetic aspects like spelling, naming, etc
> 
> IMHO, we should do whatever we can to avoid breaking them except for
> the most extreme cases.

What about the tasks API? We that discussed at the mid cycle summit and
decided that the alternative backwards compatible way of doing it was
too ugly and we didn't want to do that. But that's exactly what we'd be
doing if we implemented them in the v2 API and it would be a
feature which ends up looking bolted because of the otherwise
significant non backwards compatible API changes we can't do.

The v2 API "evolved" over a long period of time and we weren't
historically very good at reviewing the APIs before they were added. And
both we and our users are paying for it now. I think we're much better
at that now. Bumping the major version gives us the opportunity to
*eventually* get rid of that technical debt even if it costs us a bit
more in the short term.

Chris