Open Stack

On Wed, May 11, 2016, at 01:11 PM, Robert Collins wrote:
> As a community, we decided long ago to silo our code: Nova and Swift
> could have been in one tree, with multiple different artifacts - think
> of all the cross-code-base-friction we would not have had if we'd done
> that! The cultural consequence though is that bringing up new things
> is much less disruptive: add a new tree, get it configured in CI, move
> on. We're a team-centric structure for governance, and the costs of
> doing cross-team code-level initiatives are already prohibitive: we
> have already delegated all those efforts to the teams that own the
> repositories: requirements syncing, translations syncing, lint fixups,
> security audits... Having folk routinely move cross project is
> something we seem to be trying to optimise for, but we're not actually
> achieving.
> 

I think this greatly understates how much cross project work goes on. I
agree that cross-team feature development is prohibitively difficult
most of the time, but I do see a lot of cross-project reading/debugging
happening on a day to day basis. I worry that this type of work is
extremely valuable but not very visible and therefore easy to overlook.
I know I do this a lot as both a deployer and a developer, and I have to
imagine others do as well.

> So, given that that is the model - why is language part of it? Yes,
> there are minimum overheads to having a given language in CI - we need
> to be able to do robust reliable builds [or accept periodic downtime
> when the internet is not cooperating], and that sets a lower fixed
> cost, varying per language. Right now we support Python, Java,
> Javascript, Ruby in CI (as I understand it - infra focused folk please
> jump in here :)).
> 

The upfront costs are not a huge issue IMO, for reasons you hit on -
folks wanting the support for a new lanaguage are silo'd off enough that
they can pay down upfront costs without effecting the rest of the
community much. What I worry about are the maintenance costs and the
cross-team costs which are hard to quantify in a list of requirements
for a new language:

Its reasonable to assume that new languages will have similar amounts of
upstream breakage that we get from python tooling (such as a new pip
releases), and so we would just be increasing the load here by a factor
of the number of languages we gate on. This is especially concerning
given that a lot of the firefighting to solve these types of issues
seems to center around one team doing cross-project work (infra).

The ability for folks to easily debug problems across projects is a huge
issue. This isn't mostly a language issue even, its a tooling issue: we
are going to have to use and/or develop a lot of tools to replace their
python counterparts we rely on and debugging issues (especially in the
gate) is going to require knowledge of each one.

-Greg