[openstack-dev] [tempest][nova][defcore] Add option to disable some strict response checking for interop testing
Doug Hellmann
doug at doughellmann.com
Tue Jun 14 22:13:21 UTC 2016
Excerpts from Sean Dague's message of 2016-06-14 17:29:06 -0400:
> On 06/14/2016 01:57 PM, Chris Hoge wrote:
> <snip>
> >
> > My proposal for addressing this problem approaches it at two levels:
> >
> > * For the short term, I will submit a blueprint and patch to tempest that
> > allows configuration of a grey-list of Nova APIs where strict response
> > checking on additional properties will be disabled. So, for example,
> > if the 'create servers' API call returned extra properties on that call,
> > the strict checking on this line[8] would be disabled at runtime.
> > Use of this code path will emit a deprecation warning, and the
> > code will be scheduled for removal in 2017 directly after the release
> > of the 2017.01 guideline. Vendors would be required so submit the
> > grey-list of APIs with additional response data that would be
> > published to their marketplace entry.
>
> To understand more. Will there be a visible asterisk with their
> registration that says they require a grey-list?
>
> > * Longer term, vendors will be expected to work with upstream to update
> > the API for returning additional data that is compatible with
> > API micro-versioning as defined by the Nova team, and the waiver would
> > no longer be allowed after the release of the 2017.01 guideline.
> >
> > For the next half-year, I feel that this approach strengthens interoperability
> > by accurately capturing the current state of OpenStack deployments and
> > client tools. Before this change, additional properties on responses
> > weren't explicitly disallowed, and vendors and deployers took advantage
> > of this in production. While this is behavior that the Nova and QA teams
> > want to stop, it will take a bit more time to reach downstream. Also, as
> > of right now, as far as I know the only client that does strict response
> > checking for Nova responses is the Tempest client. Currently, additional
> > properties in responses are ignored and do not break existing client
> > functionality. There is currently little to no harm done to downstream
> > users by temporarily allowing additional data to be returned in responses.
>
> In general I'm ok with this, as long as three things are true:
>
> 1) registrations that need the grey list are visually indicated quite
> clearly and publicly that they needed it to pass.
I like that. Chris' proposal was that the information would need to be
submitted with the application, and I think publishing it makes sense.
I'd like to see the whole list, either which APIs had to be flagged or
at least which tests, whichever we can do.
>
> 2) 2017.01 is a firm cutoff.
>
> 3) We have evidence that folks that are having challenges with the
> strict enforcement have made getting compliant a top priority.
>
>
> 3 is the one where I don't have any data either way. But I didn't see
> any specs submissions (which are required for API changes in Nova) for
> Newton that would indicate anyone is working on this. For 2017 to be a
> hard stop, that means folks are either deleting this from their
> interface, or proposing in Ocata. Which is a really short runway if this
> stuff isn't super straight forward and already upstream agreed.
>
> So I'm provisionally ok with this, if folks in the know feel like 3 is
> covered.
>
> -Sean
>
> P.S. The Tempest changes pretty much just anticipate the Nova changes
> which are deleting all these facilities in Newton -
> https://specs.openstack.org/openstack/nova-specs/specs/newton/approved/api-no-more-extensions.html
> - so in some ways we aren't doing folks a ton of favors letting them
> delay too far because they are about to hit a brick wall on the code side.
>
> -Sean
>
More information about the OpenStack-dev
mailing list