[openstack-dev] [all] Branchless Tempest QA Spec - final draft

Sean Dague sean at dague.net
Mon Apr 28 21:31:18 UTC 2014


On 04/28/2014 02:06 PM, David Kranz wrote:
> On 04/27/2014 10:02 PM, Matthew Treinish wrote:
>> On Mon, Apr 28, 2014 at 01:01:00AM +0000, Kenichi Oomichi wrote:
>>> Hi,
>>>
>>> Sorry for my late response, but I'd like to discuss this again.
>>>
>>> Now we are working for adding Nova API responses checks to Tempest[1] to
>>> block backward incompatible changes.
>>> With this work, Tempest checks each response(status code, response body)
>>> and raises a test failure exception if detecting something unexpected.
>>> For example if some API parameter, which is defined as 'required'
>>> Tempest
>>> side, does not exist in response body, Tempest test fails.
>>>
>>> We are defining API parameters as 'required' if they are not API
>>> extensions
>>> or they are not depended on Nova configuration. In addition now Tempest
>>> allows additional API parameters, that means Tempest does not fail
>>> even if
>>> Nova response includes unexpected API parameters. Because I think the
>>> removal
>>> of API parameter causes backward incompatible issue but the addition
>>> does not
>>> cause it.
>> So, AIUI we can only add parameters to an API with a new extension.
>> The API
>> change guidelines also say that adding new properties must be
>> conditional:
>>
>> https://wiki.openstack.org/wiki/APIChangeGuidelines#Generally_Considered_OK
>>
> I just wanted to note that the original referenced wiki page, assembled
> by markmc and myself, did not specify the need for an extension in order
> to add a value to a return dict or add a value to a dict argument if the
> existing api would ignore it. This was changed (and honestly I did not
> notice this change at the time) last August to require an extension:
> https://wiki.openstack.org/w/index.php?title=APIChangeGuidelines&diff=prev&oldid=28593.
> 
> Is there any trace left of discussions around that change?
> 
> The original wording allowed the api to evolve as long as a "reasonable"
> application would not be broken. Essentially the extra value becomes
> optional and new client or server code can check for it. The new
> definition is quite strict and similar to what leads to many Windows
> APIs having names like CreateWindowEx. Not saying being strict is bad,
> but it will require a lot of changes to the client libraries as well as
> tempest because there are a lot of extensions that are not checked by
> either.

If I remember correctly that arose out of Portland summit sessions.
Basically you need to provide some signaling to the client on what they
should expect. And this kind of organically grew into that.

We pretty desperately need micro versioning so we're handling this
directly, and not via inference from extension lists.

	-Sean

-- 
Sean Dague
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 530 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140428/21ff389f/attachment.pgp>


More information about the OpenStack-dev mailing list