[openstack-dev] [all][tc] Clarifying testing recommendation for interop programs

Doug Hellmann doug at doughellmann.com
Thu Jan 18 20:36:11 UTC 2018


Excerpts from Doug Hellmann's message of 2018-01-18 15:21:12 -0500:
> Excerpts from Graham Hayes's message of 2018-01-18 19:25:02 +0000:
> > 
> > On 18/01/18 18:52, Doug Hellmann wrote:
> > > Excerpts from Graham Hayes's message of 2018-01-18 17:52:39 +0000:
> > >> On 18/01/18 16:25, Doug Hellmann wrote:
> > >>> Excerpts from Graham Hayes's message of 2018-01-18 15:33:12 +0000:
> > >>
> > >> <snip/>
> > >>
> > >>>
> > >>> In the past the QA team agreed to accept trademark-related tests from
> > >>> all projects in the tempest repo. Has that changed?
> > >>>
> > >>
> > >> There has not been an explict rejection but in all conversations the
> > >> response has been "non core projects are outside the scope of tempest".
> > >>
> > >> Honestly, everytime we have tried to do something to core tempest
> > >> we have had major pushback, and I want to clarify this before I or
> > >> someone else put in the work of porting the base clients, getting CI
> > >> configured*, and proposing the tests to tempest.
> > > 
> > > OK.
> > > 
> > > The current policy doesn't say anything about "core" or different
> > > trademark programs or any other criteria.
> > > 
> > >   The TC therefore encourages the DefCore committee to consider it an
> > >   indication of future technical direction that we do not want tests
> > >   outside of the Tempest repository used for trademark enforcement, and
> > >   that any new or existing tests that cover capabilities they want to
> > >   consider for trademark enforcement should be placed in Tempest.
> > > 
> > > That all seems very clear to me (setting aside some specific word
> > > choices like "future technical direction" that tie the resolution
> > > to language in the bylaws).  Regardless of technical reasons why
> > > it may not be necessary, we still have many social justifications
> > > for doing it the way we originally set out to do it.  Tests related
> > > to trademark enforcement need to go into the tempest repository.
> > > 
> > > The way I think this should work (and the way I remember us describing
> > > it at the time the policy was established) is the Interop WG
> > > (previously DefCore) should identify capabilities and tests, then
> > > ask project teams to reproduce those tests in the tempest repo.
> > > When the tests land, they can be used by the trademark program.
> > > Teams can also, at their leisure, decide whether to remove the
> > > original versions of the tests from whatever repo they existed in
> > > to begin with.
> > > 
> > > Graham, you've proposed a new resolution with several options for
> > > where to put tests for "add-on programs." I don't think we need
> > > that resolution if we want the tests to continue to live in tempest.
> > > The existing resolution doesn't qualify which tests, beyond "for
> > > trademark enforcement" and more words won't make that more clear,
> > > IMO.
> > > 
> > > Now if you *do* want to change the policy, we should talk about
> > > that.  But I can't tell whether you want to change it, you're worried
> > > the policy is unclear, or it is not being followed.  Can you clarify
> > > which it is?
> > 
> > It is not being followed.
> > 
> > I have brought this up at every forum session on these programs, and the
> > people in the room from QA have *always* pushed back on it.
> 
> OK, so that's a problem. I need to hear from the QA team why they've
> reversed that decision.
> 
> > 
> > And, for clarity (I saw this in a few logs) QA have *never* said that
> > they will take the interop designated tests for the DNS project into
> > openstack/tempest.
> 
> When we approved the resolution that describes the current policy, the
> QA team agreed that they would take tests for trademark. There was no
> stipulation about which projects those apply to.

I feel pretty sure that was discussed in a TC meeting, but I can't
find that. I do find Matt and Ken'ichi voting +1 on the resolution
itself.  https://review.openstack.org/#/c/312718/. If I remember
correctly, Ken'ichi was the PTL at the time.

Doug



More information about the OpenStack-dev mailing list