[openstack-dev] [Interop-wg] [QA] [PTG] [Interop] [Designate] [Heat] [TC]: QA PTG Summary- Interop test for adds-on project

Ghanshyam Mann gmann at ghanshyammann.com
Fri Mar 9 11:47:26 UTC 2018

On Fri, Mar 9, 2018 at 9:31 AM, Doug Hellmann <doug at doughellmann.com> wrote:
> Excerpts from Zane Bitter's message of 2018-03-08 15:51:05 -0500:
>> On 08/03/18 12:57, Doug Hellmann wrote:
>> > Why would the repos be owned by anyone other than the original project
>> > team?
>> A few reasons I think it makes sense in this instance:
>> * Not every set of trademark tests will necessarily belong to a single
>> project. Tempest itself is an example of this - in fact that's basically
>> how the QA program came to exist. Vertical-specific trademark programs
>> are another example that we anticipate in the future.
>> * Allowing projects to create their own repos means that there's no
>> co-ordination point to ensure e.g. a consistent naming scheme. Amongst
>> other things, this could potentially cause confusion about which plugins
>> are trademark-candidates-only and which are just regular tempest plugins.
> If these new plugins might contain "candidate" tests and all tests
> are potentially candidates, how are these new repos different from
> the existing repos that already contain all of the tests? It seems
> like at least part of the problem with the current system was
> triggered by confusion about when to move tests around to satisfy
> the policy. Can we avoid that problem with the new system? If we're
> not going to move the tests into Tempest itself and have the QA
> team manage them, why not simply take the tests from the repos where
> they already live?

I completely agree on this. If tests are going to move in Tempest
then, its all QA things and we own them completely but this is not
case now as not all projects ok to do that.
Otherwise if interop going to use tests from plugins then just consume
tests from their current location. For other future interop program
like nfv, HA etc, tests can be in new repo or if interop find any QA
projects where they can consume tests then use those QA projects
example - Extreme testing [1] which is under review though.

>> * By registering trademark plugins all in one place it makes it easy to
>> determine how many there are, which plugins exist (e.g. are there any
>> extant plugins that are not referenced by refstack? This is a question
>> you can answer in 20s if they're all registered in the same place.)
>> * The goal is for maintenance of these plugins to be a collaborative
>> effort by the project team, the QA team, and RefStack. If the first step
>> for a project establishing a trademark test plugin involves the project
>> team reaching out to the QA team then that's a good foot to start on. If
>> teams create the repos in their own projects and fly under QA's radar
>> then QA folks might not even be aware that they've become core reviewers
>> on the repo.
> I thought the QA team no longer wanted to be responsible for these
> extra tests. Has that changed again? I've lost track of everyone's
> positions, I'm afraid. Maybe we could get people to start voting
> on the actual resolutions so it's easier to keep track of that?
> As you pointed out earlier, when contributors to a repo are allowed
> to vote in the election for the team lead that owns the repo. We
> should think through the implications of that fact when we consider
> who will own these new repos (if we actually need anything new and
> we can't just use the existing repos).

For me, voting things does not matter much. What i see as overall is
that interop is ready to consume tests from different places and QA
team is all ready to share their review bandwidth with interop team,
projects team to review the interop tests irrespective of test
location and help to build process/guidelines about consistency,
do-not-change-tests etc. And we will make sure that happens.
Currently also we do such practice for tempest plugin tests (fix them,
use right interface there) and in Rocky we are going to start a
program for Tempest plugins to stabilize them where we will grep-ing
all plugins for best practice & right way setup and use interfaces.

Anyways current proposed version of resolution looks ok to me -

>> I guess we have examples of both models in the community... e.g.
>> puppet-openstack vs. Horizon plugins. I wonder if there are any lessons
>> we can draw on to see which works better, and when.
>> cheers,
>> Zane.

..1 https://review.openstack.org/#/c/443504/


> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

More information about the OpenStack-dev mailing list