Open Stack

On 05/09/2014 09:31 AM, Chicken Ohmichi wrote:
> 2014-05-09 15:00 GMT+09:00 Ghanshyam Mann <ghanshyam.mann at nectechnologies.in>:
>> Hi Sean,
>>
>>> -----Original Message-----
>>> From: Sean Dague [mailto:sean at dague.net]
>>> Sent: Monday, April 14, 2014 10:22 PM
>>> To: OpenStack Development Mailing List
>>> Subject: [openstack-dev] [all] Branchless Tempest QA Spec - final draft
>>>
>>> As we're coming up on the stable/icehouse release the QA team is looking
>>> pretty positive at no longer branching Tempest. The QA Spec draft for this is
>>> here - http://docs-draft.openstack.org/77/86577/2/check/gate-qa-specs-
>>> docs/3f84796/doc/build/html/specs/branchless-tempest.html
>>> and hopefully address a lot of the questions we've seen so far.
>>>
>>> Additional comments are welcome on the review -
>>> https://review.openstack.org/#/c/86577/
>>> or as responses on this ML thread.
>>>
>>>       -Sean
>>>
>>> --
>>> Sean Dague
>>> Samsung Research America
>>> sean at dague.net / sean.dague at samsung.com
>>> http://dague.net
>>
>> There is one more scenario where interface of the existing APIs gets changed
>> may be only for experimental APIs (Nova V3 APIs). I have following question
>> regarding those.
>>         1. How we are going to tag those in Branchless Tempest? Should we keep two versions of tests for old and new API interface.
>>         2. Till Branchless Tempest is not Implemented, tempest test
>>                      has to be skipped for those as they get failed on Ice-House gate tests.
>>                     So for tempest part of those changes should we wait for implementation of
>>                     Branchless tempest to complete?
>>
>> For example,
>>  'os-instance_actions' v3 API changed to 'os-server-actions' (https://review.openstack.org/#/c/57614/)
>>  after Ice-house release and its respective tests are skipped in tempest.
>> Now https://review.openstack.org/#/c/85666/  adds the response validation for this API in tempest.
>> As API tempest tests are skipped (cannot unskip as ice-house gate test fails), response validation code
>> will be untested on gate.
>> My question is how I should go on these-
>> 1. Should I wait for implementation of Branchless tempest to complete?
>> 2. OK to merge even this is going to be untested on gate.
> 
> I also am facing the same issue.
> The patch which fixes Nova v3 API inconsistency has been merged
> into *Nova* master, then I tried to push the other patch which blocks
> this kind of inconsistencies into *Tempest* master.
> However, check-tempest-dsvm-full-icehouse failed because of not
> backporting the Nova patch into Icehouse branch yet.
> I'm not sure we can backport this kind of patches into Icehouse
> because Nova v3 API of Icehouse must be experimental in future.
> So how about disabling v3 API tests in check-tempest-dsvm-full-icehouse?

This raises a very real question:

Why are we adding Tempest integrate tests for non stable interfaces?

My current leaning is that if you want tests in Tempest, your interface
has to be versioned. If it's not, it doesn't belong in Tempest at all.
But I think that's a good discussion point for the branchless Tempest
session in Atlanta.

	-Sean

-- 
Sean Dague
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140509/0e63399b/attachment.pgp>