Open Stack

Excerpts from Graham Hayes's message of 2018-01-18 15:33:12 +0000:

> I had hoped for more of a discussion on this before I jumped back into
> this debate  - but it seams to be stalled still, so here it goes.
> 
> I proposed this initially as we were unclear on where the tests should
> go - we had a resolution that said all tests go into openstack/tempest
> (with a list of reasons why), and the guidance and discussion that been
> had in various summits was that "add-ons" should stay in plugins.
> 
> So right now, we (by the governance rules) should be pushing tests to
> tempest for the new programs.
> 
> In the resolution that placed the tests in tempest there was a few
> reasons proposed:
> 
>   For example, API and behavioral changes must be carefully managed, as
>   must mundane aspects such as test and module naming and location
>   within the test suite. Even changes that leave tests functionally
>   equivalent may cause unexpected consequences for their use in DefCore
>   processes and validation. Placing the tests in a central repository
>   will make it easier to maintain consistency and avoid breaking the
>   trademark enforcement tool.
> 
> This still applies, and even more so for teams that traditionally do not
> have a strong QE contributor / reviewer base (aka projects not in
> "core").
> 
>   Centralizing the tests also makes it easier for anyone running the
>   validation tool against their cloud or cloud distribution to use the
>   tests. It is easier to install the test suite and its dependencies,
>   and it is easier to read and understand a set of tests following a
>   consistent implementation pattern.
> 
> Apparently users do not need central tests anymore, feedback from
> RefStack is that people who run these tests are comfortable dealing
> with extra python packages.
> 
> The point about a single set of tests, in a single location and style
> still stands.
> 
>   Finally, having the tests in a central location makes it easier to
>   ensure that all members of the community have equal input into what
>   the tests do and how they are implemented and maintained.
> 
> Seems like a good value to strive for.
> 
> One of the items that has been used to push back against adding
> "add-ons" to tempest has been that tempest has a defined scope, and
> neither of the current add-ons fit in that scope.
> 
> Can someone clarify what the set of criteria is? I think it will help
> this discussion.
> 
> Another push back is the "scaling" issue - adding more tests will
> overload the QA team.

In the past the QA team agreed to accept trademark-related tests from
all projects in the tempest repo. Has that changed?

> 
> Right now, DNS adds 10 tests, Orchestration adds 22, to a current suite
> of 353.
> 
> I do not think there is many other add-ons proposed yet, and the new
> Vertical programs will probably mainly be re-using tests in the
> openstack/tempest repos as is.
> 
> This is not a big tent-esque influx of programs - the only projects
> that can be added to the trademarks are programs in tc-approved-release
> [4], so I do not see scaling as a big issue, especially as these tests
> are such base concepts that if they need to be changed there is a
> completely new API, so the only overhead will be ensuring that nothing
> in tempest breaks the new tests (which is a good thing for trademark
> tests).
> 
> Personally, for me, I like option 3. I did not initially add it, as I
> knew it would cause endless bikesheding, but I do think it fits both
> a technical and social model.
> 
> I see 2 immediate routes forward:
> 
>  - Option 1, and we start adding these tests asap
>  - Pseudo Option 2, were we delete the resolution at [2] as it clearly
>    does not apply anymore, and abandon the review at [1].
> 
> Finally - do not conflate my actions with those of the Designate team.
> I have seen people talking about how this resolution was leverage the
> team needed to move our tests in tree. This is definitely *not* true.
> Having our tests in a plugin is useful to us, and if the above
> resolution passed, I cannot see a situation where we would try to
> move any tests that were not listed in the interop standards.
> 
> This is something I have done as an individual in the community, not
> something the designate team have pushed for.

Thanks for pushing for a clear resolution to this, Graham.

> 
> 
> [4] -
> https://governance.openstack.org/tc/reference/tags/tc_approved-release.html
> 
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >