Open Stack

Mark McLoughlin wrote:
>> * Subsystem branches with area experts are used wherever possible. The
>> subsystem maintainer should maintain this branch so that it can directly
>> be merged into master when "ready" (all or nothing). Subsystem
>> maintainers are allowed to propose a merge commit to master.
> 
> Yep, except I still think having the subsystem maintainer create and
> propose a merge commits will be a scaling issue in this process.
> [...]
> The alternative is that the maintainer of the master branch is creating
> the merge commits and resolving conflicts (like Linus):
> [...]

I tend to think we can't reproduce the kernel model as-is with
OpenStack. In particular, I don't think the PTL is willing to handle the
merge window all by himself. Our PTLs are developers elected to maintain
project goals and have the final say in technical disputes, not
necessarily people that want to spend 99% of their time resolving merge
conflicts. Linus is in a unique position, mostly due to his personal
interest and the history of Linux, to handle release management, merge
windows and have the final say. He doesn't have much time to do
development anymore.

I'm just unsure that Vish can do all of the current PTL duties (drive
project goals, resolve disputes), do what he cares about (development,
bugfixing), and handle the merge windows as well. Especially with our
subsystems that won't be as clear-cut as the kernel ones for a long
time, which means a lot more merge conflicts to solve.

That's why our proposal delegates as much work as possible to the
subsystem maintainers.

>> * Bugfixes get proposed directly to master
> 
> I think bugfixes relevant to a particular subsystem need to go through
> master, since that's where the domain experts are doing reviews.

I think you meant "through the subsystem" ? I felt the same way you did,
but Jim made a strong argument that the master branch should be the
"natural source" for branching and therefore should contain as many
bugfixes as possible. During the pre-release periods, we need to land
the fixes in master and on the release branch, and the process is
simpler if the fix lands in the release/master branch first (no
cherry-picking needed anymore).

>> * Only features that are release-ready should be accepted into master.
>> Final approval of merges on master will therefore be limited to the
>> corresponding project release team.
> 
> Woah, I didn't see that one coming :-)
> 
> Deciding what is ready for merging is the project leader's job IMHO.

Maybe you read too much into "project release team" :) It is more likely
to be the project-drivers group: a subteam of developers that are
release-quality-minded and accept to spend time on the boring task of
handling the merge window. The PTL is definitely part of this team: the
group is just a way to scale the task out for larger projects like Nova.

In all cases the PTL drives the project goals and calls the shots.

>> Benefits of this model:
>> * We enable subsystems, which for larger projects let people specialize
>> on an area of the code and avoids having to be an expert in all things
>> * Subsystems become a staging ground for features that are not release-ready
> 
> This sets off loud alarm bells for me.
> 
> A subsystem maintainer should never merge anything which could not be
> merged into master without more work. Without that rule, the subsystem
> branch gets blocked from merging into master until a feature developer
> finishes their work.
> 
> For sure, preparation work for features (or incremental steps as part of
> feature development) can be merged into a subsystem branch before the
> feature is finished. And there should be no issue with the subsystem
> maintainer proposing that for merging into master.
> 
> As an example, "replace integer IDs with UUIDs" is a feature that took
> many iterative steps. It would be crazy for a subsystem maintainer to
> merge a first step of that work into a subsystem if it couldn't be
> merged into master until the rest of the steps were finished.

Depends on what we exactly mean by subsystem. Is it a subdivision of the
code, or a topic on which several developers can collaborate ? In the
former, I agree that the subsystem would have to be kept merge-able all
the time, which would push down the task of keeping crazy stuff out to
the subsystem maintainer.

In the latter, you could have subsystems of varying degrees of
experimental-ness, as long as they are consistent. Whenever you merge
it, you need to merge it as a whole... but it could skip a release or
two before it's ready to be merged. It doesn't have to be release-ready
all the time... but everything in it should be at the same level of
release-readiness so that it can all be merged when the time comes.

I tend to prefer the latter approach that gives us more flexibility on
the "subsystems" we create. Maybe we should rename them "recognized
collaborative topic branches" ?

> If I was to pick one change I'd like to see, it would be to have the
> project leader directly responsible for merging subsystem pull requests
> and handling conflicts.
> 
> Yes, it might be hard to see how the already overburdened Vish could do
> that, but if we first do the thought experiment of what the end result
> would look like I think we'd be really eager to make it work. And I'm
> actually really confident we could do it while lightening the load on
> Vish.

I don't think the Linux kernel model is the only way to make that work.
We need to recognize that our setup is slightly different (multiple
projects having to be released at the same time, elected PTLs rather
than single dictator-for-life, Linus's interests vs. Vish's interests,
email patches vs. Gerrit changes, etc.).

We should definitely look at the kernel model as a good example, but
consider our particular model and see how we can make it scale better.
I'm not convinced channeling more work towards a single point of failure
is the best way to do it.

-- 
Thierry Carrez (ttx)
Release Manager, OpenStack