[openstack-dev] [tempest][nova][defcore] Add option to disable some strict response checking for interop testing

Doug Hellmann doug at doughellmann.com
Tue Jun 14 22:13:21 UTC 2016

Excerpts from Sean Dague's message of 2016-06-14 17:29:06 -0400:
> On 06/14/2016 01:57 PM, Chris Hoge wrote:
> <snip>
> > 
> > My proposal for addressing this problem approaches it at two levels:
> > 
> > * For the short term, I will submit a blueprint and patch to tempest that
> >   allows configuration of a grey-list of Nova APIs where strict response
> >   checking on additional properties will be disabled. So, for example,
> >   if the 'create  servers' API call returned extra properties on that call,
> >   the strict checking on this line[8] would be disabled at runtime.
> >   Use of this code path will emit a deprecation warning, and the
> >   code will be scheduled for removal in 2017 directly after the release
> >   of the 2017.01 guideline. Vendors would be required so submit the
> >   grey-list of APIs with additional response data that would be
> >   published to their marketplace entry.
> To understand more. Will there be a visible asterisk with their
> registration that says they require a grey-list?
> > * Longer term, vendors will be expected to work with upstream to update
> >   the API for returning additional data that is compatible with
> >   API micro-versioning as defined by the Nova team, and the waiver would
> >   no longer be allowed after the release of the 2017.01 guideline.
> > 
> > For the next half-year, I feel that this approach strengthens interoperability
> > by accurately capturing the current state of OpenStack deployments and
> > client tools. Before this change, additional properties on responses
> > weren't explicitly disallowed, and vendors and deployers took advantage
> > of this in production. While this is behavior that the Nova and QA teams
> > want to stop, it will take a bit more time to reach downstream. Also, as
> > of right now, as far as I know the only client that does strict response
> > checking for Nova responses is the Tempest client. Currently, additional
> > properties in responses are ignored and do not break existing client
> > functionality. There is currently little to no harm done to downstream
> > users by temporarily allowing additional data to be returned in responses.
> In general I'm ok with this, as long as three things are true:
> 1) registrations that need the grey list are visually indicated quite
> clearly and publicly that they needed it to pass.

I like that. Chris' proposal was that the information would need to be
submitted with the application, and I think publishing it makes sense.
I'd like to see the whole list, either which APIs had to be flagged or
at least which tests, whichever we can do.

> 2) 2017.01 is a firm cutoff.
> 3) We have evidence that folks that are having challenges with the
> strict enforcement have made getting compliant a top priority.
> 3 is the one where I don't have any data either way. But I didn't see
> any specs submissions (which are required for API changes in Nova) for
> Newton that would indicate anyone is working on this. For 2017 to be a
> hard stop, that means folks are either deleting this from their
> interface, or proposing in Ocata. Which is a really short runway if this
> stuff isn't super straight forward and already upstream agreed.
> So I'm provisionally ok with this, if folks in the know feel like 3 is
> covered.
>     -Sean
> P.S. The Tempest changes pretty much just anticipate the Nova changes
> which are deleting all these facilities in Newton -
> https://specs.openstack.org/openstack/nova-specs/specs/newton/approved/api-no-more-extensions.html
> - so in some ways we aren't doing folks a ton of favors letting them
> delay too far because they are about to hit a brick wall on the code side.
>     -Sean

More information about the OpenStack-dev mailing list