[openstack-dev] [all][tc] Clarifying testing recommendation for interop programs

Graham Hayes gr at ham.ie
Thu Jan 18 14:48:30 UTC 2018



On 12/01/18 09:49, Luigi Toscano wrote:
> On Thursday, 11 January 2018 23:52:00 CET Matt Riedemann wrote:
>> On 1/11/2018 10:36 AM, Colleen Murphy wrote:
>>> 1) All trademark-related tests should go in the tempest repo, in
>>> accordance
>>>
>>>     with the original resolution. This would mean that even projects that
>>>     have
>>>     never had tests in tempest would now have to add at least some of
>>>     their
>>>     black-box tests to tempest.
>>>
>>> The value of this option is that centralizes tests used for the Interop
>>> program in a location where interop-minded folks from the QA team can
>>> control them. The downside is that projects that so far have avoided
>>> having a dependency on tempest will now lose some control over the
>>> black-box tests that they use for functional and integration that would
>>> now also be used for trademark certification.
>>> There's also concern for the review bandwidth of the QA team - we can't
>>> expect the QA team to be continually responsible for an ever-growing list
>>> of projects and their trademark tests.
>>
>> How many tests are we talking about for designate and heat? Half a
>> dozen? A dozen? More?
>>
>> If it's just a couple of tests per project it doesn't seem terrible to
>> have them live in Tempest so you get the "interop eye" on reviews, as
>> noted in your email. If it's a considerable amount, then option 2 seems
>> the best for the majority of parties.
> 
> I would argue that it does not scale; what if some test is taken out from the 
> interoperability, and others are added? It would mean moving tests from one 
> repository to another, with change of paths. I think that the solution 2, 
> where the repository where a test belong and the functionality of a test are 
> not linked, is better.
> 
> Ciao
> 

How is this different from the 6 programs already in tempest?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180118/4e44566b/attachment.sig>


More information about the OpenStack-dev mailing list