Open Stack

It's really easy to just say "don't break the contract." Until we got
the level of testing that we currently have in Tempest, the contract was
broken pretty regularly. I'm sure there are still breaks in it around
the edges where we aren't clamping down on people today.

So the history of v2 is far from being a stable API in the traditional
sense.

Which isn't to say we're trying to go and make the whole thing fluid.
However there has to be a path forward for incremental improvement,
because there are massive short comings in the existing API.

While a big bang approach might work for smaller interfaces, the Nova
API surface is huge. So huge, it's not even fully documented. Which
means we're at a state where you aren't implementing to an API, you are
implementing to an implementation. And if you look at HP and RAX you'll
find enough differences to make you scratch your head a bunch. And
that's only 2 data points. I'm sure the private cloud products have all
kinds of funkiness in them.

So we do really need to be pragmatic here as well. Because our
experience with v3 so far has been doing a major version bump on Nova is
a minimum of 2 years, and that doesn't reach a completion point that
anyone's happy with to switch over.

So, that begs a new approach. Because I think at this point even if we
did put out Nova v3, there can never be a v4. It's too much, too big,
and doesn't fit in the incremental nature of the project. So whatever
gets decided about v3, the thing that's important to me is a sane way to
be able to add backwards compatible changes (which we actually don't
have today, and I don't think any other service in OpenStack does
either), as well a mechanism for deprecating parts of the API. With some
future decision about whether removing them makes sense.

	-Sean

On 02/24/2014 05:01 PM, Morgan Fainberg wrote:
> On the topic of backwards incompatible changes:
> 
> I strongly believe that breaking current clients that use the APIs
> directly is the worst option possible. All the arguments about needing
> to know which APIs work based upon which backend drivers are used are
> all valid, but making an API incompatible change when we’ve made the
> contract that the current API will be stable is a very bad approach.
> Breaking current clients isn’t just breaking “novaclient", it would also
> break any customers that are developing directly against the API. In the
> case of cloud deployments with real-world production loads on them (and
> custom development around the APIs) upgrading between major versions is
> already difficult to orchestrate (timing, approvals, etc), if we add in
> the need to re-work large swaths of code due to API changes, it will
> become even more onerous and perhaps drive deployers to forego the
> upgrades in lieu of stability.
> 
> If the perception is that we don’t have stable APIs (especially when we
> are ostensibly versioning them), driving adoption of OpenStack becomes
> significantly more difficult. Difficulty in driving further adoption
> would be a big negative to both the project and the community.
> 
> TL;DR, “don’t break the contract”. If we are seriously making
> incompatible changes (and we will be regardless of the direction) the
> only reasonable option is a new major version.
> 
> *—*
> *Morgan Fainberg*
> Principal Software Engineer
> Core Developer, Keystone
> m at metacloud.com <mailto:m at metacloud.com>
> 
> 
> On February 24, 2014 at 10:16:31, Matt Riedemann
> (mriedem at linux.vnet.ibm.com <mailto://mriedem@linux.vnet.ibm.com>) wrote:
> 
>>
>>
>> On 2/24/2014 10:13 AM, Russell Bryant wrote:
>> > On 02/24/2014 01:50 AM, Christopher Yeoh wrote:
>> >> Hi,
>> >>
>> >> There has recently been some speculation around the V3 API and whether
>> >> we should go forward with it or instead backport many of the changes
>> >> to the V2 API. I believe that the core of the concern is the extra
>> >> maintenance and test burden that supporting two APIs means and the
>> >> length of time before we are able to deprecate the V2 API and return
>> >> to maintaining only one (well two including EC2) API again.
>> >
>> > Yes, this is a major concern.  It has taken an enormous amount of work
>> > to get to where we are, and v3 isn't done.  It's a good time to
>> > re-evaluate whether we are on the right path.
>> >
>> > The more I think about it, the more I think that our absolute top goal
>> > should be to maintain a stable API for as long as we can reasonably do
>> > so.  I believe that's what is best for our users.  I think if you gave
>> > people a choice, they would prefer an inconsistent API that works for
>> > years over dealing with non-backwards compatible jumps to get a nicer
>> > looking one.
>> >
>> > The v3 API and its unit tests are roughly 25k lines of code.  This also
>> > doesn't include the changes necessary in novaclient or tempest.  That's
>> > just *our* code.  It explodes out from there into every SDK, and then
>> > end user apps.  This should not be taken lightly.
>> >
>> >> This email is rather long so here's the TL;DR version:
>> >>
>> >> - We want to make backwards incompatible changes to the API
>> >>    and whether we do it in-place with V2 or by releasing V3
>> >>    we'll have some form of dual API support burden.
>> >>    - Not making backwards incompatible changes means:
>> >>      - retaining an inconsistent API
>> >
>> > I actually think this isn't so bad, as discussed above.
>> >
>> >>      - not being able to fix numerous input validation issues
>> >
>> > I'm not convinced, actually.  Surely we can do a lot of cleanup here.
>> > Perhaps you have some examples of what we couldn't do in the existing API?
>> >
>> > If it's a case of wanting to be more strict, some would argue that the
>> > current behavior isn't so bad (see robustness principle [1]):
>> >
>> >      "Be conservative in what you do, be liberal in what you accept from
>> >      others (often reworded as "Be conservative in what you send, be
>> >      liberal in what you accept")."
>> >
>> > There's a decent counter argument to this, too.  However, I still fall
>> > back on it being best to just not break existing clients above all else.
>> >
>> >>      - have to forever proxy for glance/cinder/neutron with all
>> >>        the problems that entails.
>> >
>> > I don't think I'm as bothered by the proxying as others are.  Perhaps
>> > it's not architecturally pretty, but it's worth it to maintain
>> > compatibility for our users.
>>
>> +1 to this, I think this is also related to what Jay Pipes is saying in
>> his reply:
>>
>> "Whether a provider chooses to, for example,
>> deploy with nova-network or Neutron, or Xen vs. KVM, or support block
>> migration for that matter *should have no effect on the public API*. The
>> fact that those choices currently *do* effect the public API that is
>> consumed by the client is a major indication of the weakness of the API."
>>
>> As a consumer, I don't want to have to know which V2 APIs work and which
>> don't depending on if I'm using nova-network or Neutron.
>>
>> >
>> >>    - Backporting V3 infrastructure changes to V2 would be a
>> >>      considerable amount of programmer/review time
>> >
>> > Agreed, but so is the ongoing maintenance and development of v3.
>> >
>> >>
>> >> - The V3 API as-is has:
>> >>    - lower maintenance
>> >>    - is easier to understand and use (consistent).
>> >>    - Much better input validation which is baked-in (json-schema)
>> >>      rather than ad-hoc and incomplete.
>> >
>> > So here's the rub ... with the exception of the consistency bits, none
>> > of this is visible to users, which makes me think we should be able to
>> > do all of this on v2.
>> >
>> >> - Whilst we have existing users of the API we also have a lot more
>> >>    users in the future. It would be much better to allow them to use
>> >>    the API we want to get to as soon as possible, rather than trying
>> >>    to evolve the V2 API and forcing them along the transition that they
>> >>    could otherwise avoid.
>> >
>> > I'm not sure I understand this.  A key point is that I think any
>> > evolving of the V2 API has to be backwards compatible, so there's no
>> > forcing them along involved.
>> >
>> >> - We already have feature parity for the V3 API (nova-network being
>> >>    the exception due to the very recent unfreezing of it), novaclient
>> >>    support, and a reasonable transition path for V2 users.
>> >>
>> >> - Proposed way forward:
>> >>    - Release the V3 API in Juno with nova-network and tasks support
>> >>    - Feature freeze the V2 API when the V3 API is released
>> >>      - Set the timeline for deprecation of V2 so users have a lot
>> >>        of warning
>> >>      - Fallback for those who really don't want to move after
>> >>    deprecation is an API service which translates between V2 and V3
>> >>    requests, but removes the dual API support burden from Nova.
>> >
>> > One of my biggest principles with a new API is that we should not have
>> > to force a migration with a strict timeline like this.  If we haven't
>> > built something compelling enough to get people to *want* to migrate as
>> > soon as they are able, then we haven't done our job.  Deprecation of the
>> > old thing should only be done when we feel it's no longer wanted or used
>> > by the vast majority.  I just don't see that happening any time soon.
>> >
>> > We have a couple of ways forward right now.
>> >
>> > 1) Continue as we have been, and plan to release v3 once we have a
>> > compelling enough feature set.
>> >
>> > 2) Take what we have learned from v3 and apply it to v2.  For example:
>> >
>> >   - The plugin infrastructure is an internal implementation detail that
>> >     can be done with the existing API.
>> >
>> >   - extension versioning is a concept we can add to v2
>> >
>> >   - we've also been discussing the concept of a core minor version, to
>> >     reflect updates to the core that are backwards compatible.  This
>> >     seems doable in v2.
>> >
>> >   - revisit a new major API when we get to the point of wanting to
>> >     effectively do a re-write, where we are majorly re-thinking the
>> >     way our API is designed (from an external perspective, not internal
>> >     implementation).
>> >
>> > [1] http://en.wikipedia.org/wiki/Robustness_principle
>> >
>>
>> -- 
>>
>> Thanks,
>>
>> Matt Riedemann
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140224/3cbf4c7e/attachment.pgp>