[openstack-dev] [all][tc] Clarifying testing recommendation for interop programs

Graham Hayes gr at ham.ie
Thu Jan 18 19:25:02 UTC 2018

On 18/01/18 18:52, Doug Hellmann wrote:
> Excerpts from Graham Hayes's message of 2018-01-18 17:52:39 +0000:
>> On 18/01/18 16:25, Doug Hellmann wrote:
>>> Excerpts from Graham Hayes's message of 2018-01-18 15:33:12 +0000:
>> <snip/>
>>> In the past the QA team agreed to accept trademark-related tests from
>>> all projects in the tempest repo. Has that changed?
>> There has not been an explict rejection but in all conversations the
>> response has been "non core projects are outside the scope of tempest".
>> Honestly, everytime we have tried to do something to core tempest
>> we have had major pushback, and I want to clarify this before I or
>> someone else put in the work of porting the base clients, getting CI
>> configured*, and proposing the tests to tempest.
> OK.
> The current policy doesn't say anything about "core" or different
> trademark programs or any other criteria.
>   The TC therefore encourages the DefCore committee to consider it an
>   indication of future technical direction that we do not want tests
>   outside of the Tempest repository used for trademark enforcement, and
>   that any new or existing tests that cover capabilities they want to
>   consider for trademark enforcement should be placed in Tempest.
> That all seems very clear to me (setting aside some specific word
> choices like "future technical direction" that tie the resolution
> to language in the bylaws).  Regardless of technical reasons why
> it may not be necessary, we still have many social justifications
> for doing it the way we originally set out to do it.  Tests related
> to trademark enforcement need to go into the tempest repository.
> The way I think this should work (and the way I remember us describing
> it at the time the policy was established) is the Interop WG
> (previously DefCore) should identify capabilities and tests, then
> ask project teams to reproduce those tests in the tempest repo.
> When the tests land, they can be used by the trademark program.
> Teams can also, at their leisure, decide whether to remove the
> original versions of the tests from whatever repo they existed in
> to begin with.
> Graham, you've proposed a new resolution with several options for
> where to put tests for "add-on programs." I don't think we need
> that resolution if we want the tests to continue to live in tempest.
> The existing resolution doesn't qualify which tests, beyond "for
> trademark enforcement" and more words won't make that more clear,
> IMO.
> Now if you *do* want to change the policy, we should talk about
> that.  But I can't tell whether you want to change it, you're worried
> the policy is unclear, or it is not being followed.  Can you clarify
> which it is?

It is not being followed.

I have brought this up at every forum session on these programs, and the
people in the room from QA have *always* pushed back on it.

And, for clarity (I saw this in a few logs) QA have *never* said that
they will take the interop designated tests for the DNS project into

To the point that the interop tooling was developed to support plugins
(which would seem to be in breach of this resolution, but I am sure
there is reasons for this.)

I do want to have option 3 (interop-tempest-plugin), but right now I
will settle for us either:

A: Doing what we planned on before (Option 1) (Prefered)
B: Documenting the fact that things have changed (Option 2), and 	
   articulate and record the reasoning for the change.

I think Add Ons are going to the Board in Dublin for the change from
Advisory, in the 2018.02 standard so we need to get clarity on this.

- Graham

> Doug
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180118/8898ca36/attachment.sig>

More information about the OpenStack-dev mailing list