[openstack-dev] [all] Branchless Tempest QA Spec - final draft

Christopher Yeoh cbkyeoh at gmail.com
Tue Apr 29 02:41:26 UTC 2014


On Tue, Apr 29, 2014 at 7:01 AM, Sean Dague <sean at dague.net> wrote:

> On 04/28/2014 02:06 PM, David Kranz wrote:
> > On 04/27/2014 10:02 PM, Matthew Treinish wrote:
> >> On Mon, Apr 28, 2014 at 01:01:00AM +0000, Kenichi Oomichi wrote:
> >>> Hi,
> >>>
> >>> Sorry for my late response, but I'd like to discuss this again.
> >>>
> >>> Now we are working for adding Nova API responses checks to Tempest[1]
> to
> >>> block backward incompatible changes.
> >>> With this work, Tempest checks each response(status code, response
> body)
> >>> and raises a test failure exception if detecting something unexpected.
> >>> For example if some API parameter, which is defined as 'required'
> >>> Tempest
> >>> side, does not exist in response body, Tempest test fails.
> >>>
> >>> We are defining API parameters as 'required' if they are not API
> >>> extensions
> >>> or they are not depended on Nova configuration. In addition now Tempest
> >>> allows additional API parameters, that means Tempest does not fail
> >>> even if
> >>> Nova response includes unexpected API parameters. Because I think the
> >>> removal
> >>> of API parameter causes backward incompatible issue but the addition
> >>> does not
> >>> cause it.
> >> So, AIUI we can only add parameters to an API with a new extension.
> >> The API
> >> change guidelines also say that adding new properties must be
> >> conditional:
> >>
> >>
> https://wiki.openstack.org/wiki/APIChangeGuidelines#Generally_Considered_OK
> >>
> > I just wanted to note that the original referenced wiki page, assembled
> > by markmc and myself, did not specify the need for an extension in order
> > to add a value to a return dict or add a value to a dict argument if the
> > existing api would ignore it. This was changed (and honestly I did not
> > notice this change at the time) last August to require an extension:
> >
> https://wiki.openstack.org/w/index.php?title=APIChangeGuidelines&diff=prev&oldid=28593
> .
> >
> > Is there any trace left of discussions around that change?
> >
> > The original wording allowed the api to evolve as long as a "reasonable"
> > application would not be broken. Essentially the extra value becomes
> > optional and new client or server code can check for it. The new
> > definition is quite strict and similar to what leads to many Windows
> > APIs having names like CreateWindowEx. Not saying being strict is bad,
> > but it will require a lot of changes to the client libraries as well as
> > tempest because there are a lot of extensions that are not checked by
> > either.
>
> If I remember correctly that arose out of Portland summit sessions.
> Basically you need to provide some signaling to the client on what they
> should expect. And this kind of organically grew into that.
>
>
Yes, as can be seen in the Nova V2 API code this convention of adding a new
extension when adding a parameter
has been happening for quite a while even if not explicitly documented. I
think it is a reasonable expectation that a client be able to tell if a
feature
is supported or not rather than just guessing based on what data is
returned.

We pretty desperately need micro versioning so we're handling this

> directly, and not via inference from extension lists.
>
>

Microversioning is something we should consider, but with the V3 API at
least the explicit versioning for on extensions is sufficient. Explicit
extension versioning
for say V2.1 is also a possibility.

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140429/1e9fa0f7/attachment.html>


More information about the OpenStack-dev mailing list