[openstack-dev] [all][tc] Clarifying testing recommendation for interop programs

Graham Hayes gr at ham.ie
Fri Jan 19 10:57:01 UTC 2018

On 19/01/18 00:52, Ken'ichi Ohmichi wrote:
> 2018-01-18 12:36 GMT-08:00 Doug Hellmann <doug at doughellmann.com>:
>> Excerpts from Doug Hellmann's message of 2018-01-18 15:21:12 -0500:
>>> Excerpts from Graham Hayes's message of 2018-01-18 19:25:02 +0000:
>>>> On 18/01/18 18:52, Doug Hellmann wrote:
>>>>> Excerpts from Graham Hayes's message of 2018-01-18 17:52:39 +0000:
>>>>>> On 18/01/18 16:25, Doug Hellmann wrote:
>>>>>>> Excerpts from Graham Hayes's message of 2018-01-18 15:33:12 +0000:
>>>>>> <snip/>
>>>>>>> In the past the QA team agreed to accept trademark-related tests from
>>>>>>> all projects in the tempest repo. Has that changed?
>>>>>> There has not been an explict rejection but in all conversations the
>>>>>> response has been "non core projects are outside the scope of tempest".
>>>>>> Honestly, everytime we have tried to do something to core tempest
>>>>>> we have had major pushback, and I want to clarify this before I or
>>>>>> someone else put in the work of porting the base clients, getting CI
>>>>>> configured*, and proposing the tests to tempest.
>>>>> OK.
>>>>> The current policy doesn't say anything about "core" or different
>>>>> trademark programs or any other criteria.
>>>>>   The TC therefore encourages the DefCore committee to consider it an
>>>>>   indication of future technical direction that we do not want tests
>>>>>   outside of the Tempest repository used for trademark enforcement, and
>>>>>   that any new or existing tests that cover capabilities they want to
>>>>>   consider for trademark enforcement should be placed in Tempest.
>>>>> That all seems very clear to me (setting aside some specific word
>>>>> choices like "future technical direction" that tie the resolution
>>>>> to language in the bylaws).  Regardless of technical reasons why
>>>>> it may not be necessary, we still have many social justifications
>>>>> for doing it the way we originally set out to do it.  Tests related
>>>>> to trademark enforcement need to go into the tempest repository.
>>>>> The way I think this should work (and the way I remember us describing
>>>>> it at the time the policy was established) is the Interop WG
>>>>> (previously DefCore) should identify capabilities and tests, then
>>>>> ask project teams to reproduce those tests in the tempest repo.
>>>>> When the tests land, they can be used by the trademark program.
>>>>> Teams can also, at their leisure, decide whether to remove the
>>>>> original versions of the tests from whatever repo they existed in
>>>>> to begin with.
>>>>> Graham, you've proposed a new resolution with several options for
>>>>> where to put tests for "add-on programs." I don't think we need
>>>>> that resolution if we want the tests to continue to live in tempest.
>>>>> The existing resolution doesn't qualify which tests, beyond "for
>>>>> trademark enforcement" and more words won't make that more clear,
>>>>> IMO.
>>>>> Now if you *do* want to change the policy, we should talk about
>>>>> that.  But I can't tell whether you want to change it, you're worried
>>>>> the policy is unclear, or it is not being followed.  Can you clarify
>>>>> which it is?
>>>> It is not being followed.
>>>> I have brought this up at every forum session on these programs, and the
>>>> people in the room from QA have *always* pushed back on it.
>>> OK, so that's a problem. I need to hear from the QA team why they've
>>> reversed that decision.
>>>> And, for clarity (I saw this in a few logs) QA have *never* said that
>>>> they will take the interop designated tests for the DNS project into
>>>> openstack/tempest.
>>> When we approved the resolution that describes the current policy, the
>>> QA team agreed that they would take tests for trademark. There was no
>>> stipulation about which projects those apply to.
>> I feel pretty sure that was discussed in a TC meeting, but I can't
>> find that. I do find Matt and Ken'ichi voting +1 on the resolution
>> itself.  https://review.openstack.org/#/c/312718/. If I remember
>> correctly, Ken'ichi was the PTL at the time.
> Yeah, I have still agreed with the resolution.
> When I voted +1 on that, core projects were defined as 6 projects like
> Nova, Cinder, Glance, Keystone, Neutron and Swift.
> And the project navigator also showed these 6 projects as core projects.
> Now I cannot find such definition on the project navigator[1], the
> definition has been changed?
> I just want to clarify "is it true that designate and heat become core
> projects?"
> If there is a concrete decision, I don't have any objections against
> that we have these projects tests in Tempest as the resolution.

This seems to be the problem - there is not now, or ever been a "core"
project definition that was decided by TC / community. We have a set of
projects that most people will refer to as "core", but there is no way
to add projects to that.

What was highlighted on the project navigator was a set of projects the
marketing dept in the foundation considered core, which is definitely
*not* something we as a community should use as a technical basis for

> Thanks
> Ken Ohmichi
> ---
> [1]: https://www.openstack.org/software/project-navigator
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180119/8d675285/attachment.sig>

More information about the OpenStack-dev mailing list