Open Stack

On Wed, 31 May 2017, Doug Hellmann wrote:
> Yeah, it sounds like the current organization of the repo is not
> ideal in terms of equal playing field for all of our project teams.
> I would be fine with all of the interop tests being in a plugin
> together, or of saying that the tempest repo should only contain
> those tests and that others should move to their own plugins. If we're
> going to reorganize all of that, we should decide what new structure we
> want and work it into the goal.

I feel like the discussion about the interop tests has strayed this
conversation from the more general point about plugin "fairness" and
allowed the vagueness in plans for interop to control our thinking
and discussion about options in the bigger view.

<blink>
This is pretty standard for an OpenStack conversation:

* introduce a general idea or critique
* someone latches on to one small aspect of that idea that presents
   some challenges, narrowing the context
* that latching and those challenges is used to kill the introspection
   that the general idea was pursuing, effectively killing any
   opportunities for learning and discovery that could lead to
   improvement or innovation

This _has_ to stop. We're at my three year anniversary in the
community and this has been and still is my main concern with the
how we collaborate. There is so much stop energy and chilling effect
in the way we discuss things in OpenStack. So much fatigue over
issues being brought up "over and over again" or things being
discussed without immediate solutions in mind. So what! Time moves
forward which means the context for issues is always changing.
Discussion is how we identify problems! Discussion is how we
get good solutions! 
</blink>

It's clear from this thread and other conversations that the
management of tempest plugins is creating a multiplicity of issues
and confusions:

* Some projects are required to use plugins and some are not. This
   creates classes of projects.

* Those second class projects now need to move their plugins to
   other repos because rules.

* Projects with plugins need to put their tests in their new repos,
   except for some special tests which will be identified by a vague
   process.

* Review of changes is intermittent and hard to track because
   stakeholders need to think about multiple locations, without
   guidance.

* People who want to do validation with tempest need to gather stuff
   from a variety of locations.

* Tempest is a hammer used for lots of different nails, but the
   type of nail varies over time and with the whimsy of policy.

* Discussion of using something other than tempest for interop is
   curtailed by policy which appears to be based in "that's the way
   it is done".

A lot of this results, in part, from there being no single guiding
pattern and principle for how (and where) the tests are to be
managed. When there's a choice between one, some and all, "some" is
almost always the wrong way to manage something. "some" is how we do
tempest (and fair few other OpenStack things).

If it is the case that we want some projects to not put their tests
in the main tempest repo then the only conceivable pattern from a
memorability, discoverability, and equality standpoint is actually
for all the tests to be in plugins.

If that isn't possible (and it is clear there are many reasons why
that may be the case) then we need to be extra sure that we explore
and uncover the issues that the "some" approach presents and provide
sufficient documentation, tooling, and guidance to help people get
around them. And that we recognize and acknowledge the impact it has.

If the answer to that is "who is going to do that?" or "who has the
time?" then I ask you to ask yourself why we think the "non-core"
projects have time to fiddle about with tempest plugins?

And finally, I actually don't have too strong of a position in the
case of tempest and tempest plugins. What I take issue with is the
process whereby we discuss and decide these things and characterize
the various projects.

If I have any position on tempest at all it is that we should limit
it to gross cloud validation and maybe interop testing, and projects
should manage their own integration testing in tree using whatever
tooling they feel is most appropriate. If that turns out to be
tempest, cool.

-- 
Chris Dent                  ┬──┬◡ﾉ(° -°ﾉ)       https://anticdent.org/
freenode: cdent                                         tw: @anticdent