Open Stack

On 2014-09-24 13:55:57 -0400 (-0400), Zane Bitter wrote:
[...]
> * Assumption #2: Yawnoc's Law
> 
> Don't bother Googling that, I just made it up. It's the reverse of
> Conway's Law:
> 
>   Infra engineers who design governance structures for OpenStack
>   are constrained to produce designs that are copies of the
>   structure of Tempest.
> 
> I just don't understand why that needs to be the case. Currently,
> for understandable historic reasons, every project gates against
> every other project. That makes no sense any more, completely
> independently of the project governance structure. We should just
> change it! There is no organisational obstacle to changing how
> gating works.
[...]

Note that to a great extent the current proliferation of integration
testing was driven from the developers of many projects. The Infra
and QA teams didn't just wake up one morning and decide to chuck all
the projects in. Rewind a year or two and remember that we had
massive amounts of asymmetry in our testing. Project A would
implement some new change and Project B developers would get their
jobs insta-broken, then come complaining that clearly this meant we
should be gating Project A on whether or not Project B works.

For a time we had sufficient (human and server) resources to do
that, and so it was comparatively cheap to just keep expanding the
list of projects which shared a common set of jobs we hoped
exercised enough of everything to act as a canary in the coal mine.
We're now running into pretty clear scalability limits on the number
of development teams whose changes you can effectively test against
each other given the realities of nondeterministic failures,
breakdowns in cross-group communication, growth in size of
integrated releases, "tragedy of the commons" effects on
horizontal efforts/shared resources, external factors like
dependency changes, et cetera.

As a community we should explore solutions (and clearly are) to
these underlying problems, but also need to reconsider some old
habits that need changing such as tight coupling to the internal
APIs and implementation details of other projects... especially if
doing so lets us scale back our integration testing, rather than
leaning on it harder so that these undesirable development patterns
can continue unabated.
-- 
Jeremy Stanley