Open Stack

Excerpts from Chris Hoge's message of 2016-06-14 10:57:05 -0700:
> Last year, in response to Nova micro-versioning and extension updates[1],
> the QA team added strict API schema checking to Tempest to ensure that
> no additional properties were added to Nova API responses[2][3]. In the
> last year, at least three vendors participating the the OpenStack Powered
> Trademark program have been impacted by this change, two of which
> reported this to the DefCore Working Group mailing list earlier this year[4].
> 
> The DefCore Working Group determines guidelines for the OpenStack Powered
> program, which includes capabilities with associated functional tests
> from Tempest that must be passed, and designated sections with associated
> upstream code [5][6]. In determining these guidelines, the working group
> attempts to balance the future direction of development with lagging
> indicators of deployments and user adoption.
> 
> After a tremendous amount of consideration, I believe that the DefCore
> Working Group needs to implement a temporary waiver for the strict API
> checking requirements that were introduced last year, to give downstream
> deployers more time to catch up with the strict micro-versioning
> requirements determined by the Nova/Compute team and enforced by the
> Tempest/QA team.
> 
> My reasoning behind this is that while the change that enabled strict
> checking was discussed publicly in the developer community and took
> some time to be implemented, it still landed quickly and broke several
> existing deployments overnight. As Tempest has moved forward with
> bug and UX fixes (some in part to support the interoperability testing
> efforts of the DefCore Working Group), using an older versions of Tempest
> where this strict checking is not enforced is no longer a viable solution
> for downstream deployers. The TC has passed a resolution to advise
> DefCore to use Tempest as the single source of capability testing[7],
> but this naturally introduces tension between the competing goals of
> maintaining upstream functional testing and also tracking lagging
> indicators.
> 
> My proposal for addressing this problem approaches it at two levels:
> 
> * For the short term, I will submit a blueprint and patch to tempest that
>   allows configuration of a grey-list of Nova APIs where strict response
>   checking on additional properties will be disabled. So, for example,
>   if the 'create  servers' API call returned extra properties on that call,
>   the strict checking on this line[8] would be disabled at runtime.
>   Use of this code path will emit a deprecation warning, and the
>   code will be scheduled for removal in 2017 directly after the release
>   of the 2017.01 guideline. Vendors would be required so submit the
>   grey-list of APIs with additional response data that would be
>   published to their marketplace entry.
> 
> * Longer term, vendors will be expected to work with upstream to update
>   the API for returning additional data that is compatible with
>   API micro-versioning as defined by the Nova team, and the waiver would
>   no longer be allowed after the release of the 2017.01 guideline.
> 
> For the next half-year, I feel that this approach strengthens interoperability
> by accurately capturing the current state of OpenStack deployments and
> client tools. Before this change, additional properties on responses
> weren't explicitly disallowed, and vendors and deployers took advantage
> of this in production. While this is behavior that the Nova and QA teams
> want to stop, it will take a bit more time to reach downstream. Also, as
> of right now, as far as I know the only client that does strict response
> checking for Nova responses is the Tempest client. Currently, additional
> properties in responses are ignored and do not break existing client
> functionality. There is currently little to no harm done to downstream
> users by temporarily allowing additional data to be returned in responses.

Thanks for putting this proposal together, Chris. The configuration
option you describe makes sense as a temporary solution to the
issue, and the timeline you propose (combined with the past year
since the change went in) should be plenty of time to handle upgrades.

Doug

> 
> Thanks,
> 
> Chris Hoge
> Interop Engineer
> OpenStack Foundation
> 
> [1] https://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/api-microversions.html
> [2] http://lists.openstack.org/pipermail/openstack-dev/2015-February/057613.html
> [3] https://review.openstack.org/#/c/156130
> [4] http://lists.openstack.org/pipermail/defcore-committee/2016-January/000986.html
> [5] http://git.openstack.org/cgit/openstack/defcore/tree/2015.07.json
> [6] http://git.openstack.org/cgit/openstack/defcore/tree/2016.01.json
> [7] http://git.openstack.org/cgit/openstack/governance/tree/resolutions/20160504-defcore-test-location.rst
> [8] http://git.openstack.org/cgit/openstack/tempest-lib/tree/tempest_lib/api_schema/response/compute/v2_1/servers.py#n39
>