Open Stack

On the topic of backwards incompatible changes:

I strongly believe that breaking current clients that use the APIs directly is the worst option possible. All the arguments about needing to know which APIs work based upon which backend drivers are used are all valid, but making an API incompatible change when we’ve made the contract that the current API will be stable is a very bad approach. Breaking current clients isn’t just breaking “novaclient", it would also break any customers that are developing directly against the API. In the case of cloud deployments with real-world production loads on them (and custom development around the APIs) upgrading between major versions is already difficult to orchestrate (timing, approvals, etc), if we add in the need to re-work large swaths of code due to API changes, it will become even more onerous and perhaps drive deployers to forego the upgrades in lieu of stability.

If the perception is that we don’t have stable APIs (especially when we are ostensibly versioning them), driving adoption of OpenStack becomes significantly more difficult. Difficulty in driving further adoption would be a big negative to both the project and the community.

TL;DR, “don’t break the contract”. If we are seriously making incompatible changes (and we will be regardless of the direction) the only reasonable option is a new major version.
—
Morgan Fainberg
Principal Software Engineer
Core Developer, Keystone
m at metacloud.com


On February 24, 2014 at 10:16:31, Matt Riedemann (mriedem at linux.vnet.ibm.com) wrote:



On 2/24/2014 10:13 AM, Russell Bryant wrote:  
> On 02/24/2014 01:50 AM, Christopher Yeoh wrote:  
>> Hi,  
>>  
>> There has recently been some speculation around the V3 API and whether  
>> we should go forward with it or instead backport many of the changes  
>> to the V2 API. I believe that the core of the concern is the extra  
>> maintenance and test burden that supporting two APIs means and the  
>> length of time before we are able to deprecate the V2 API and return  
>> to maintaining only one (well two including EC2) API again.  
>  
> Yes, this is a major concern. It has taken an enormous amount of work  
> to get to where we are, and v3 isn't done. It's a good time to  
> re-evaluate whether we are on the right path.  
>  
> The more I think about it, the more I think that our absolute top goal  
> should be to maintain a stable API for as long as we can reasonably do  
> so. I believe that's what is best for our users. I think if you gave  
> people a choice, they would prefer an inconsistent API that works for  
> years over dealing with non-backwards compatible jumps to get a nicer  
> looking one.  
>  
> The v3 API and its unit tests are roughly 25k lines of code. This also  
> doesn't include the changes necessary in novaclient or tempest. That's  
> just *our* code. It explodes out from there into every SDK, and then  
> end user apps. This should not be taken lightly.  
>  
>> This email is rather long so here's the TL;DR version:  
>>  
>> - We want to make backwards incompatible changes to the API  
>> and whether we do it in-place with V2 or by releasing V3  
>> we'll have some form of dual API support burden.  
>> - Not making backwards incompatible changes means:  
>> - retaining an inconsistent API  
>  
> I actually think this isn't so bad, as discussed above.  
>  
>> - not being able to fix numerous input validation issues  
>  
> I'm not convinced, actually. Surely we can do a lot of cleanup here.  
> Perhaps you have some examples of what we couldn't do in the existing API?  
>  
> If it's a case of wanting to be more strict, some would argue that the  
> current behavior isn't so bad (see robustness principle [1]):  
>  
> "Be conservative in what you do, be liberal in what you accept from  
> others (often reworded as "Be conservative in what you send, be  
> liberal in what you accept")."  
>  
> There's a decent counter argument to this, too. However, I still fall  
> back on it being best to just not break existing clients above all else.  
>  
>> - have to forever proxy for glance/cinder/neutron with all  
>> the problems that entails.  
>  
> I don't think I'm as bothered by the proxying as others are. Perhaps  
> it's not architecturally pretty, but it's worth it to maintain  
> compatibility for our users.  

+1 to this, I think this is also related to what Jay Pipes is saying in  
his reply:  

"Whether a provider chooses to, for example,  
deploy with nova-network or Neutron, or Xen vs. KVM, or support block  
migration for that matter *should have no effect on the public API*. The  
fact that those choices currently *do* effect the public API that is  
consumed by the client is a major indication of the weakness of the API."  

As a consumer, I don't want to have to know which V2 APIs work and which  
don't depending on if I'm using nova-network or Neutron.  

>  
>> - Backporting V3 infrastructure changes to V2 would be a  
>> considerable amount of programmer/review time  
>  
> Agreed, but so is the ongoing maintenance and development of v3.  
>  
>>  
>> - The V3 API as-is has:  
>> - lower maintenance  
>> - is easier to understand and use (consistent).  
>> - Much better input validation which is baked-in (json-schema)  
>> rather than ad-hoc and incomplete.  
>  
> So here's the rub ... with the exception of the consistency bits, none  
> of this is visible to users, which makes me think we should be able to  
> do all of this on v2.  
>  
>> - Whilst we have existing users of the API we also have a lot more  
>> users in the future. It would be much better to allow them to use  
>> the API we want to get to as soon as possible, rather than trying  
>> to evolve the V2 API and forcing them along the transition that they  
>> could otherwise avoid.  
>  
> I'm not sure I understand this. A key point is that I think any  
> evolving of the V2 API has to be backwards compatible, so there's no  
> forcing them along involved.  
>  
>> - We already have feature parity for the V3 API (nova-network being  
>> the exception due to the very recent unfreezing of it), novaclient  
>> support, and a reasonable transition path for V2 users.  
>>  
>> - Proposed way forward:  
>> - Release the V3 API in Juno with nova-network and tasks support  
>> - Feature freeze the V2 API when the V3 API is released  
>> - Set the timeline for deprecation of V2 so users have a lot  
>> of warning  
>> - Fallback for those who really don't want to move after  
>> deprecation is an API service which translates between V2 and V3  
>> requests, but removes the dual API support burden from Nova.  
>  
> One of my biggest principles with a new API is that we should not have  
> to force a migration with a strict timeline like this. If we haven't  
> built something compelling enough to get people to *want* to migrate as  
> soon as they are able, then we haven't done our job. Deprecation of the  
> old thing should only be done when we feel it's no longer wanted or used  
> by the vast majority. I just don't see that happening any time soon.  
>  
> We have a couple of ways forward right now.  
>  
> 1) Continue as we have been, and plan to release v3 once we have a  
> compelling enough feature set.  
>  
> 2) Take what we have learned from v3 and apply it to v2. For example:  
>  
> - The plugin infrastructure is an internal implementation detail that  
> can be done with the existing API.  
>  
> - extension versioning is a concept we can add to v2  
>  
> - we've also been discussing the concept of a core minor version, to  
> reflect updates to the core that are backwards compatible. This  
> seems doable in v2.  
>  
> - revisit a new major API when we get to the point of wanting to  
> effectively do a re-write, where we are majorly re-thinking the  
> way our API is designed (from an external perspective, not internal  
> implementation).  
>  
> [1] http://en.wikipedia.org/wiki/Robustness_principle  
>  

--  

Thanks,  

Matt Riedemann  


_______________________________________________  
OpenStack-dev mailing list  
OpenStack-dev at lists.openstack.org  
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140224/fb6b8acb/attachment.html>