[openstack-dev] [all][tc] Clarifying testing recommendation for interop programs

Graham Hayes gr at ham.ie
Thu Jan 18 19:25:02 UTC 2018



On 18/01/18 18:52, Doug Hellmann wrote:
> Excerpts from Graham Hayes's message of 2018-01-18 17:52:39 +0000:
>> On 18/01/18 16:25, Doug Hellmann wrote:
>>> Excerpts from Graham Hayes's message of 2018-01-18 15:33:12 +0000:
>>
>> <snip/>
>>
>>>
>>> In the past the QA team agreed to accept trademark-related tests from
>>> all projects in the tempest repo. Has that changed?
>>>
>>
>> There has not been an explict rejection but in all conversations the
>> response has been "non core projects are outside the scope of tempest".
>>
>> Honestly, everytime we have tried to do something to core tempest
>> we have had major pushback, and I want to clarify this before I or
>> someone else put in the work of porting the base clients, getting CI
>> configured*, and proposing the tests to tempest.
> 
> OK.
> 
> The current policy doesn't say anything about "core" or different
> trademark programs or any other criteria.
> 
>   The TC therefore encourages the DefCore committee to consider it an
>   indication of future technical direction that we do not want tests
>   outside of the Tempest repository used for trademark enforcement, and
>   that any new or existing tests that cover capabilities they want to
>   consider for trademark enforcement should be placed in Tempest.
> 
> That all seems very clear to me (setting aside some specific word
> choices like "future technical direction" that tie the resolution
> to language in the bylaws).  Regardless of technical reasons why
> it may not be necessary, we still have many social justifications
> for doing it the way we originally set out to do it.  Tests related
> to trademark enforcement need to go into the tempest repository.
> 
> The way I think this should work (and the way I remember us describing
> it at the time the policy was established) is the Interop WG
> (previously DefCore) should identify capabilities and tests, then
> ask project teams to reproduce those tests in the tempest repo.
> When the tests land, they can be used by the trademark program.
> Teams can also, at their leisure, decide whether to remove the
> original versions of the tests from whatever repo they existed in
> to begin with.
> 
> Graham, you've proposed a new resolution with several options for
> where to put tests for "add-on programs." I don't think we need
> that resolution if we want the tests to continue to live in tempest.
> The existing resolution doesn't qualify which tests, beyond "for
> trademark enforcement" and more words won't make that more clear,
> IMO.
> 
> Now if you *do* want to change the policy, we should talk about
> that.  But I can't tell whether you want to change it, you're worried
> the policy is unclear, or it is not being followed.  Can you clarify
> which it is?

It is not being followed.

I have brought this up at every forum session on these programs, and the
people in the room from QA have *always* pushed back on it.

And, for clarity (I saw this in a few logs) QA have *never* said that
they will take the interop designated tests for the DNS project into
openstack/tempest.

To the point that the interop tooling was developed to support plugins
(which would seem to be in breach of this resolution, but I am sure
there is reasons for this.)

I do want to have option 3 (interop-tempest-plugin), but right now I
will settle for us either:

A: Doing what we planned on before (Option 1) (Prefered)
B: Documenting the fact that things have changed (Option 2), and 	
   articulate and record the reasoning for the change.

I think Add Ons are going to the Board in Dublin for the change from
Advisory, in the 2018.02 standard so we need to get clarity on this.

- Graham

> Doug
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180118/8898ca36/attachment.sig>


More information about the OpenStack-dev mailing list