[openstack-dev] [nova] stable branches & failure to handle review backlog

Ihar Hrachyshka ihrachys at redhat.com
Mon Aug 18 10:39:18 UTC 2014

Hash: SHA512

On 18/08/14 11:00, Thierry Carrez wrote:
> Mark McLoughlin wrote:
>> [...] I don't see how any self-respecting open-source project can
>> throw a release over the wall and have no ability to address
>> critical bugs with that release until the next release 6 months
>> later which will also include a bunch of new feature work with
>> new bugs. That's not a distro maintainer point of view.
> I agree that the job changed a bit since those early days, and now
> apart from a very small group of stable specialists (mostly the
> stable release managers), everyone else on stable-core is actually
> specialized in a given project. It would make sense for each
> project to have a set of dedicated "stable liaisons" that would
> work together with stable release managers in getting critical
> bugfixes to stable branch point releases. Relying on the same small
> group of people now that we have 10+ projects to cover is
> unreasonable.

Indeed, not everyone feels safe when reviewing stable backports for
projects he is not really involved in. For example, I'm more or less
fine with Neutron and Oslo, and some Nova patches, but once it gets to
e.g. Horizon or Keystone, I feel very worried that I even have +2 for
those branches, and generally don't apply it unless I'm completely
sure the fix is obvious, and the bug is clear for outsider (=me).

The problem is that some projects lack people that are both confident
in the code base and are dedicated to support stable maint effort. So
while Neutron or Nova do not feel great deficit in the number of
maintainers who care about their stable branches, for other projects
it may be really hard to find anyone to review the code.

Dedicated liasons would solve that.

> There are two issues to solve before we do that, though. The
> projects have to be OK with taking on that extra burden (it becomes
> their responsibility to dedicate resources to stable branches). And
> we need to make sure the stable branch guidelines are well
> communicated.
>> [...] That's quite a leap to say that -core team members will be
>> so incapable of the appropriate level of conservatism that the
>> branch will be killed.
>> The idea behind not automatically granting +2 on stable/* to
>> -core members was simply we didn't want people diving in and
>> approving unsuitable stuff out of ignorance.
>> I could totally see an argument now that everyone is a lot more
>> familiar with gerrit, the concept of stable releases is well
>> established and we should be able to trust -core team members to
>> learn how to make the risk vs benefit tradeoffs needed for stable
>> reviews.
> The question here is whether every -core member on a project
> should automatically be on stable-core (and we can reuse the same
> group) or do we have to maintain two groups.
> From my experience reviewing stable branch changes, I see two types
> of issues with regular core doing stable reviews. There is the
> accidental stable/* review, where the person thinks he is reviewing
> a master review. Gerrit makes it look extremely similar, so more
> often than not, we have -core members wondering why they don't have
> +2 on review X, or core members doing a code review (criticizing
> the code again) rather than a stable review (checking the type of
> change and the backport). Maybe we could solve that by making
> gerrit visually different on stable reviews ?

I would be glad if stable patches are not shown by default in queries
unless explicitly requested. That would solve the issue of people
reviewing the code of backports, and others.

I would also like to have some way to explicitly mark a stable branch
patch as being 'not a trivial cherry-pick'. Sometimes stable branch
patches are not really similar to their original master fellows. Or
even we need to push a stable branch only patch because the issue or
the affected code is not in master. In that case, stable maintainers
are not the ones to do review, because the code should be checked by
the real cores of the affected projects, and checking technical
details and general applicability of the patch for stable branch is
not enough. Something like a button to mark the backport as requiring
special attention from cores.

Both the split and a way to drive cores' attention to specific
backport would remove burden from both project cores and stable
maintainers that sometimes struggle to review non-trivial bug fixes.

> And then there is the opportunistic review, where a core member
> bends the stable rules because he wants a specific fix backported.
> So we get database migrations, new configuration options, or
> default behavior changes +1ed or +2ed in stable. When we discuss
> those on the review, the -core person generally disagrees with the
> stable rules and would like them relaxed.
> This is why I tend to prefer an opt-in system where the core
> member signs up to do stable review. He clearly says he agrees with
> the stable rules and will follow them. He also signs up to do
> enough of them to limit the opportunistic and accidental issues.
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)


More information about the OpenStack-dev mailing list