Open Stack

So this thread is getting deep again, as I expect they all will, so I'm
just going to top post and take the ire for doing so.

I also want to summarize what I've seen in the threads so far:

v2 needed forever - if I do a sentiment analysis here looking at the
orgs people are coming from, most of this is currently coming from
Rackspace folks (publicly). There might be many reasons for this, one of
which is the fact that they've done a big transition in the near past
(between *not openstack* and Openstack), and felt that pain.
Understanding that pain is good.

It is interesting that Phil actually brings up a completely different
issue from the HP cloud side, which is the amount of complaints they are
having to field about how terrible the v2 API is. HP has actually had an
OpenStack cloud public longer than Rackspace. So this feedback shouldn't
be lost.

So I feel like while some deployers have expressed no interest in moving
forward on API, others can't get there soon enough.

Which makes me think a lot about point 4. As has already been suggested
we could actually make v2 a proxy to v3. And just like with images and
volumes, it becomes frozen in Juno, and people that want more features
will move to the v3 API. Just like with other services.

This requires internal cleanups as well. However it wouldn't shut down
future evolution of the interface.

Nova really has 4 interfaces today
 * Nova v2 JSON
 * Nova v2 XML
 * Nova v3 JSON
 * EC2

I feel like if we had to lose one to decrease maintenance cost, Nova v2
XML is the one to lose. And if we did, v2 on v3 isn't the craziest thing
in the world. It's not free, but neither is the backport.

I also feel like the code duplication approach for v2 and v3 was taken
because the Glance folks had a previously bad experience on common code
with 2 APIs. However, their surface was small enough that the dual code
trees were fine. Nova is a beast in this regard.

I also feel like more folks are excited about working on the v2 on v3
approach, given that Kenichi is already prototyping on it. And it is
important to have excitement in this work as well. The human factor,
about the parts that people want to work on, is critical to the success
of Nova as a project.

This project has always been about coming up with the best ideas,
regardless of where they came from. So I'd like to make sure people
aren't making up their mind early on this, and have retreated to the
idea of winning this discussion. The outcome of this is important.
Because it basically sets up the framework of how we work on improving
the Nova interface... for a long time.

I also feel like the 2 complaints that come up on v3 repeatedly, that
it's a bunch of copy and paste, and that it doesn't implement anything
new, are kind of disingenuous. Because those were explicit design
constraints imposed and agreed on by the nova core team on the approach.
And if those are the complaints on the approach, why weren't those
brought up in advance?

At the end of the day, I'm on the fence. Because what I actually want is
somewhat orthogonal to all of this. Which is a real discoverable
interface, and substantially less optional Nova core. And I've yet to
figure out which approach sets us up for doing that better.

	-Sean

On 03/03/2014 12:32 PM, Russell Bryant wrote:
> There has been quite a bit of discussion about the future of the v3 API
> recently.  There has been growing support for the idea that we should
> change course and focus on evolving the existing v2 API instead of
> putting out a new major revision.  This message is a more complete
> presentation of that proposal that concludes that we can do what we
> really need to do with only the v2 API.
> 
> Keeping only the v2 API requires some confidence that we can stick with
> it for years to come.  We don't want to be revisiting this any time
> soon.  This message addresses a bunch of different questions about how
> things would work if we only had v2.
> 
> 1) What about tasks?
> 
> In some cases, the proposed integration of tasks is backwards
> compatible.  A task ID will be added to a header.  The biggest point of
> debate was if and how we would change the response for creating a
> server.  For tasks in v2, we would not change the response by default.
> The task ID would just be in a header.  However, if and when the client
> starts exposing version support information, we can provide an
> alternative/preferred response based on tasks.
> 
> For example:
> 
>    Accept: application/json;type=task
> 
> 2) Versioning extensions
> 
> One of the points being addressed in the v3 API was the ability to
> version extensions.  In v2, we have historically required new API
> extensions, even for changes that are backwards compatible.  We propose
> the following:
> 
>  - Add a version number to v2 API extensions
>  - Allow backwards compatible changes to these API extensions,
> accompanied by a version number increase
>  - Add the option to advertise an extension as deprecated, which can be
> used for all those extensions created only to advertise the availability
> of new input parameters
> 
> 3) Core versioning
> 
> Another pain point in API maintenance has been having to create API
> extensions for every small addition to the core API.  We propose that a
> version number be exposed for the core API that exposes the revision of
> the core API in use.  With that in place, backwards compatible changes
> such as adding a new property to a resource would be allowed when
> accompanied by a version number increase.
> 
> With versioning of the core and API extensions, we will be able to cut
> down significantly on the number of changes that require an API
> extension without sacrificing the ability of a client to discover
> whether the addition is present or not.
> 
> 4) API Proxying
> 
> We don't see proxying APIs as a problem.  It is the cost we pay for
> choosing to split apart projects after they are released.  We don't
> think it's fair to break users just because we have chosen to split
> apart the backend implementation.
> 
> Further, the APIs that are proxied are frozen while those in the other
> projects are evolving.  We believe that as more features are available
> only via the native APIs in Cinder, Glance, and Neutron, users will
> naturally migrate over to the native APIs.
> 
> Over time, we can ensure clients are able to query the API without the
> need to proxy by adding new formats or extensions that don't return data
> that needed to be proxied.
> 
> 5) Capitalization and Naming Consistency
> 
> Some of the changes in the v3 API included changes to capitalization and
> naming for improved consistency.  If we stick with v2 only, we will not
> be able to make any of these changes.  However, we believe that not
> breaking any existing clients and not having to maintain a second API is
> worth not making these changes, or supporting some indirection to
> achieve this for newer clients if we decide it is important.
> 
> 6) Response Code Consistency and Correctness
> 
> The v2 API has many places where the response code returned for a given
> operation is not strictly correct. For example a 200 is returned when a
> 202 would be more appropriate. Correcting these issues should be
> considered for improving the future use of the API, however there does
> not seem to be any support for considering this a critical problem right
> now. There are two approaches that can be taken to improve this in v2:
> 
> Just fix them. Right now, we return some codes that imply we have dealt
> with a request, when all we have done is queue it for processing (and
> vice versa). In the future, we may change the backend in such a way that
> a return code needs to change to continue to be accurate anyway. If we
> just assume that return codes may change to properly reflect the action
> that was taken, then we can correct existing errors and move on.
> Accept them as wrong but not critically so. With this approach, we can
> strive for correctness in the future without changing behavior of our
> existing APIs. Nobody seems to complain about them right now, so
> changing them seems to be little gain. If the client begins exposing a
> version header (which we need for other things) then we could
> alternately start returning accurate codes for those clients.
> 
> The key point here is that we see a way forward with this in the v2 API
> regardless of which path we choose.
> 
> 7) Entrypoint based extensions
> 
> The v3 effort included improvements to the infrastructure used to
> implement the API, both for proper extensions and modular construction
> of the core API.  We definitely want that for v2, and since these are
> just internal implementation details, there is no reason why we can't
> implement these improvements in the v2 API.  Note that with the addition
> of versioning of extensions and the core, we should be adding fewer API
> extensions and may be able to collapse some that we already have.
> 
> 8) Input Validation
> 
> Previously, much of our input validation happened at the database driver
> layer for various reasons. This means that behavior of the API depends
> on such user-invisible things as which RDBM is in use. Thus, our input
> validation is already inconsistent across deployments. Further, the move
> to objects as the communication mechanism between the API and the
> backend means that more input validation is going on than we once had
> anyway, and this is not something we can avoid unless we freeze the
> backend anyway.
> 
> Exposing a definition like jsonschema is something that we should do
> anyway, and the process of doing so should allow us to define what was
> previously undefined. Given the variance of the behavior depending on
> the database used on the backend, getting something reasonably strict
> should be roughly equivalent (user-wise) to working against a provider
> that was using something other than MySQL.
> 
> 9) v3 ... if not now, when?
> 
> Not in the foreseeable future.  We don't see it happening unless some
> group of people wanted to rebuild the API from scratch in such a way
> that it doesn't look anything like the current API.  The new API would
> have to be different and compelling enough that we'd be willing to
> maintain it along side the current API for several years.
> 
> 10) Conclusion and Proposal
> 
> The v3 API effort has produced a lot of excellent work.  However, the
> majority opinion seems to be that we should avoid the cost of
> maintaining two APIs if at all possible.  We should apply what has been
> learned to the existing API where we can and focus on making v2
> something that we can continue to maintain for years to come.
> 
> We recognize and accept that it is a failure of Nova project leadership
> that we did not come to this conclusion much sooner.  We hope to have
> learned from the experience to help avoiding a situation like this
> happening again in the future.
> 
> Please provide your input on this proposal, even if it is just agreement
> with this as the way forward.
> 
> Thanks,
> 
> Proposal authors / sponsors:
> 
>     Russell Bryant
>     Dan Smith
>     John Garbutt
>     Andrew Laski
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140304/9f0b623c/attachment.pgp>