[openstack-dev] [tempest][nova][defcore] Add option to disable some strict response checking for interop testing

Sean Dague sean at dague.net
Tue Jun 14 21:29:06 UTC 2016

On 06/14/2016 01:57 PM, Chris Hoge wrote:
> My proposal for addressing this problem approaches it at two levels:
> * For the short term, I will submit a blueprint and patch to tempest that
>   allows configuration of a grey-list of Nova APIs where strict response
>   checking on additional properties will be disabled. So, for example,
>   if the 'create  servers' API call returned extra properties on that call,
>   the strict checking on this line[8] would be disabled at runtime.
>   Use of this code path will emit a deprecation warning, and the
>   code will be scheduled for removal in 2017 directly after the release
>   of the 2017.01 guideline. Vendors would be required so submit the
>   grey-list of APIs with additional response data that would be
>   published to their marketplace entry.

To understand more. Will there be a visible asterisk with their
registration that says they require a grey-list?

> * Longer term, vendors will be expected to work with upstream to update
>   the API for returning additional data that is compatible with
>   API micro-versioning as defined by the Nova team, and the waiver would
>   no longer be allowed after the release of the 2017.01 guideline.
> For the next half-year, I feel that this approach strengthens interoperability
> by accurately capturing the current state of OpenStack deployments and
> client tools. Before this change, additional properties on responses
> weren't explicitly disallowed, and vendors and deployers took advantage
> of this in production. While this is behavior that the Nova and QA teams
> want to stop, it will take a bit more time to reach downstream. Also, as
> of right now, as far as I know the only client that does strict response
> checking for Nova responses is the Tempest client. Currently, additional
> properties in responses are ignored and do not break existing client
> functionality. There is currently little to no harm done to downstream
> users by temporarily allowing additional data to be returned in responses.

In general I'm ok with this, as long as three things are true:

1) registrations that need the grey list are visually indicated quite
clearly and publicly that they needed it to pass.

2) 2017.01 is a firm cutoff.

3) We have evidence that folks that are having challenges with the
strict enforcement have made getting compliant a top priority.

3 is the one where I don't have any data either way. But I didn't see
any specs submissions (which are required for API changes in Nova) for
Newton that would indicate anyone is working on this. For 2017 to be a
hard stop, that means folks are either deleting this from their
interface, or proposing in Ocata. Which is a really short runway if this
stuff isn't super straight forward and already upstream agreed.

So I'm provisionally ok with this, if folks in the know feel like 3 is


P.S. The Tempest changes pretty much just anticipate the Nova changes
which are deleting all these facilities in Newton -
- so in some ways we aren't doing folks a ton of favors letting them
delay too far because they are about to hit a brick wall on the code side.


Sean Dague

More information about the OpenStack-dev mailing list