[openstack-dev] [tempest][nova][defcore] Add option to disable some strict response checking for interop testing

Mark Voelker mvoelker at vmware.com
Wed Jun 15 13:45:36 UTC 2016

> On Jun 15, 2016, at 8:01 AM, Sean Dague <sean at dague.net> wrote:
> On 06/15/2016 12:14 AM, Mark Voelker wrote:
> <snip>
>> It is perhaps important to note here that the DefCore seems to have two meanings to a lot of people I talk to today: it’s a mark of interoperability (the OpenStack Powered badge that says certain capabilities of this cloud behave like other clouds bearing the mark) and it gives a cloud the ability to call itself OpenStack (e.g. you can get a trademark/logo license agreement from the Foundation).  
>> The OpenStack Powered program currently covers Icehouse through Mitaka.  Right now, that includes releases that were still on the Nova 2.0 API.  API extensions were a supported thing [1] back in 2.0 and it was even explicitly documented that they allowed for additional attributes in the responses and “vendor specific niche functionality [1]”.  The change to the Tempest tests [2] applied to the 2.0 API as well as 2.1 with the intent of preventing further changes from getting into the 2.0 API at the gate, which totally makes sense as a gate test.  If those same tests are used for DefCore purposes, it does change what vendors need to do to be compliant with the Guidelines rather immediately--even on older releases of OpenStack using 2.0, which could be problematic (as noted elsewhere already [3]).
> Right, that's fair. And part of why I think the pass* makes sense.
> Liberty is the introduction of microversions on by default for clouds
> from Nova upstream configs.
>> So, through the interoperability lens: I think many folks acknowledge that supporting extensions lead to a lot of variance between clouds, and that was Not So Awesome for interoperability.  IIRC part of the rationale for switching to microversions with a single monotonic counter and deprecating extensions [4] was to set a course for eliminating a lot of that behavioral variance.
>> From the “ability to call yourself OpenStack” lens: it feels sort of wrong to tell a cloud that it can’t claim to be OpenStack because it’s running a version that falls within the bounds of the Powered program with the 2.0 API (when extensions weren't deprecated) and using the extension mechanism that 2.0 supported for years.
> To be clear, extensions weren't a part of the 2.0 API, they were a part
> of the infrastructure. It's a subtle but different point. Nova still
> supports the 2.0 API, but on different infrastructure, which
> doesn't/won't support extensions.
> Are people registering new Kilo (or earlier) clouds in the system today?
> By the time folks get to Newton, none of that is going to work anyway in
> code.

Yes.  As recently as January we had a Juno-based public cloud run into this issue [1].  And, just to highlight again: there's a pretty long tail here since the OpenStack Powered program covers a lot of releases.  

[1] http://lists.openstack.org/pipermail/defcore-committee/2016-January/000986.html

> In an ideal world product teams would be close enough to upstream code
> and changes to see all this coming and be on top of it. In the real
> world, a lot of these teams are like a year (or more) behind, which

+1.  If folks aren’t aware, it may be worth pointing out that as of the User Survey from two months ago, Kilo was the most popular release and Juno was the third most popular [2].  I don’t presume to know why so many deployments are on older releases, but I suspect the rationale is different for different folks.  Maybe for some it’s because people are choosing to used products and a lot of those are based on older releases.  Maybe for others it’s because people are deliberately choosing older stable branches.  Maybe for some it’s because they they’ve decided that upgrading every six months or following master more closely just isn’t a good strategy for them.  Maybe for others it’s something else entirely.  But again, you’re right: in the real world the tail is long today.

[2] https://www.openstack.org/assets/survey/April-2016-User-Survey-Report.pdf (figure 4.3/4.4)

> actually makes Defcore with a pass* an ideal alternative communication
> channel to express that products are coming up to a cliff, and should
> start working on plans now.


>> I think that’s part of what makes this issue tricky for a lot of folks.
>> [1] http://docs.openstack.org/developer/nova/v2/extensions.html
> It's an unfortunate accident of bugs in our publishing system that that
> URL still exists. That was deleted in Oct 2015. I'll look at getting it
> properly cleaned up.

Awesome, thanks Sean!  I’ll make a point to comb around and see if there are other such references that we should clean up too. 

Just to be clear though, my main point isn’t that the doc still exists today, it’s that it existed.  In the relatively recent past (from a long-tail perspective) extensions were documented as an acceptable mechanism for vendor specific functionality, and that makes me a little bit more inclined to provide some additional runway for clouds that actually used that mechanism so they can get to a more interoperable place.  It’s great that we’ve recognized the old mechanism created problems and I think it’s awesome that we’re moving to a model that’s more interoperable.  I want our entire ecosystem to get to a more interoperable place--the mechanics of getting a big ecosystem there just seem to be taking a little more time than I, for one, might have hoped a few months ago.

At Your Service,

Mark T. Voelker

> 	-Sean
> -- 
> Sean Dague
> https://urldefense.proofpoint.com/v2/url?u=http-3A__dague.net&d=CwIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=Q8IhPU-EIzbG5YDx5LYO7zEJpGZykn7RwFg-UTPWvDc&m=YCAObeSeyHQTzTvV6NEv47PRp3ZrIGN5cBA-aN9P_sA&s=2aJMubnViInBp3WchmKFv-m5zq98iRQnRZdDY_3XUIM&e= 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

More information about the OpenStack-dev mailing list