[openstack-dev] [all][tc] Clarifying testing recommendation for interop programs

Doug Hellmann doug at doughellmann.com
Thu Jan 18 18:52:17 UTC 2018

Excerpts from Graham Hayes's message of 2018-01-18 17:52:39 +0000:
> On 18/01/18 16:25, Doug Hellmann wrote:
> > Excerpts from Graham Hayes's message of 2018-01-18 15:33:12 +0000:
> <snip/>
> > 
> > In the past the QA team agreed to accept trademark-related tests from
> > all projects in the tempest repo. Has that changed?
> > 
> There has not been an explict rejection but in all conversations the
> response has been "non core projects are outside the scope of tempest".
> Honestly, everytime we have tried to do something to core tempest
> we have had major pushback, and I want to clarify this before I or
> someone else put in the work of porting the base clients, getting CI
> configured*, and proposing the tests to tempest.


The current policy doesn't say anything about "core" or different
trademark programs or any other criteria.

  The TC therefore encourages the DefCore committee to consider it an
  indication of future technical direction that we do not want tests
  outside of the Tempest repository used for trademark enforcement, and
  that any new or existing tests that cover capabilities they want to
  consider for trademark enforcement should be placed in Tempest.

That all seems very clear to me (setting aside some specific word
choices like "future technical direction" that tie the resolution
to language in the bylaws).  Regardless of technical reasons why
it may not be necessary, we still have many social justifications
for doing it the way we originally set out to do it.  Tests related
to trademark enforcement need to go into the tempest repository.

The way I think this should work (and the way I remember us describing
it at the time the policy was established) is the Interop WG
(previously DefCore) should identify capabilities and tests, then
ask project teams to reproduce those tests in the tempest repo.
When the tests land, they can be used by the trademark program.
Teams can also, at their leisure, decide whether to remove the
original versions of the tests from whatever repo they existed in
to begin with.

Graham, you've proposed a new resolution with several options for
where to put tests for "add-on programs." I don't think we need
that resolution if we want the tests to continue to live in tempest.
The existing resolution doesn't qualify which tests, beyond "for
trademark enforcement" and more words won't make that more clear,

Now if you *do* want to change the policy, we should talk about
that.  But I can't tell whether you want to change it, you're worried
the policy is unclear, or it is not being followed.  Can you clarify
which it is?


More information about the OpenStack-dev mailing list