[openstack-dev] [Interop-wg] [QA] [PTG] [Interop] [Designate] [Heat] [TC]: QA PTG Summary- Interop test for adds-on project
zbitter at redhat.com
Fri Mar 9 15:06:25 UTC 2018
On 08/03/18 19:31, Doug Hellmann wrote:
> If these new plugins might contain "candidate" tests and all tests
> are potentially candidates, how are these new repos different from
> the existing repos that already contain all of the tests?
We have to allow candidate tests because otherwise there's a
chicken-and-egg type problem where we can never add tests to an existing
trademark program. But the definition of 'candidate' that I'm thinking
of is along the lines of "under active discussion to be added to the
next iteration of refstack". Perhaps that should be documented.
To answer your question directly, the main way they're different is that
by putting tests in one of these repos, teams are committing to not
really change them, and certainly to ~never break backwards compat.
> It seems
> like at least part of the problem with the current system was
> triggered by confusion about when to move tests around to satisfy
> the policy. Can we avoid that problem with the new system? If we're
> not going to move the tests into Tempest itself and have the QA
> team manage them, why not simply take the tests from the repos where
> they already live?
I think a lesson we've learned from Tempest is that there are two parts
to successfully reviewing a change:
1) Determine whether the change affects a test that is part of the
2) Don't make the change if it does.
2 is easy, but 1 is hard. By separating the trademark tests into a
separate repo, we make 1 easy as well. This reduces risk and increases
> I thought the QA team no longer wanted to be responsible for these
> extra tests. Has that changed again? I've lost track of everyone's
> positions, I'm afraid.
Their position as I understand it is:
* They're not going to write tests
* They're happy to document the process, offer advice, and review tests
as time allows
* They don't want tests to be thrown over the transom and made their
problem for the rest of time
That doesn't conflict with anybody's goals here, so it's mostly a matter
of documenting it clearly so that somebody reading without all this
context won't get the wrong idea.
More information about the OpenStack-dev