Open Stack

> -----Original Message-----
> From: Matthew Treinish [mailto:mtreinish at kortar.org]
> Sent: Tuesday, November 10, 2015 3:12 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [stable] Making stable maintenance its own
> OpenStack project team
> 
> On Mon, Nov 09, 2015 at 10:54:43PM +0000, Kuvaja, Erno wrote:
> > > On Mon, Nov 09, 2015 at 05:28:45PM -0500, Doug Hellmann wrote:
> > > > Excerpts from Matt Riedemann's message of 2015-11-09 16:05:29 -0600:
> > > > >
> > > > > On 11/9/2015 10:41 AM, Thierry Carrez wrote:
> > > > > > Hi everyone,
> > > > > >
> > > > > > A few cycles ago we set up the Release Cycle Management team
> > > > > > which was a bit of a frankenteam of the things I happened to be
> leading:
> > > > > > release management, stable branch maintenance and
> > > > > > vulnerability
> > > management.
> > > > > > While you could argue that there was some overlap between
> > > > > > those functions (as in, "all these things need to be
> > > > > > released") logic was not the primary reason they were put together.
> > > > > >
> > > > > > When the Security Team was created, the VMT was spinned out of
> > > > > > the Release Cycle Management team and joined there. Now I
> > > > > > think we should spin out stable branch maintenance as well:
> > > > > >
> > > > > > * A good chunk of the stable team work used to be stable point
> > > > > > release management, but as of stable/liberty this is now done
> > > > > > by the release management team and triggered by the
> > > > > > project-specific stable maintenance teams, so there is no more
> > > > > > overlap in tooling used there
> > > > > >
> > > > > > * Following the kilo reform, the stable team is now focused on
> > > > > > defining and enforcing a common stable branch policy[1],
> > > > > > rather than approving every patch. Being more visible and
> > > > > > having more dedicated members can only help in that very
> > > > > > specific mission
> > > > > >
> > > > > > * The release team is now headed by Doug Hellmann, who is
> > > > > > focused on release management and does not have the history I
> > > > > > had with stable branch policy. So it might be the right moment
> > > > > > to refocus release management solely on release management and
> > > > > > get the stable team its own leadership
> > > > > >
> > > > > > * Empowering that team to make its own decisions, giving it
> > > > > > more visibility and recognition will hopefully lead to more
> > > > > > resources being dedicated to it
> > > > > >
> > > > > > * If the team expands, it could finally own stable branch
> > > > > > health and gate fixing. If that ends up all falling under the
> > > > > > same roof, that team could make decisions on support
> > > > > > timeframes as well, since it will be the primary resource to
> > > > > > make that work
> > > > >
> > > > > Isn't this kind of already what the stable maint team does?
> > > > > Well, that and some QA people like mtreinish and sdague.
> > > > >
> > > > > >
> > > > > > So.. good idea ? bad idea ? What do current
> > > > > > stable-maint-core[2] members think of that ? Who thinks they
> > > > > > could step up to lead that
> > > team ?
> > > > > >
> > > > > > [1]
> > > > > > http://docs.openstack.org/project-team-guide/stable-branches.h
> > > > > > tml [2]
> > > > > > https://review.openstack.org/#/admin/groups/530,members
> > > > > >
> > > > >
> > > > > With the decentralizing of the stable branch stuff in Liberty
> > > > > [1] it seems like there would be less use for a PTL for stable
> > > > > branch maintenance - the cats are now herding themselves, right?
> > > > > Or at least that's the plan as far as I understood it. And the
> > > > > existing stable branch wizards are more or less around for help
> > > > > and answering
> > > questions.
> > > >
> > > > The same might be said about releasing from master and the release
> > > > management team. There's still some benefit to having people
> > > > dedicated to making sure projects all agree to sane policies and
> > > > to keep up with deliverables that need to be released.
> > >
> > > Except the distinction is that relmgt is actually producing
> > > something. Relmgt has the releases repo which does centralize
> > > library releases, reno to do the release notes, etc. What does the
> > > global stable core do? Right now it's there almost entirely to just add
> people to the project specific stable core teams.
> > >
> > > -Matt Treinish
> >
> >
> > I'd like to move the discussion from what are the roles of the current
> stable-maint-core and more towards what the benefits would be having a
> stable-maint team rather than the -core group alone.
> >
> > Personally I think the stable maintenance should be quite a lot more than
> unblocking gate and approving people allowed to merge to the stable
> branches.
> >
> 
> Sure, but that's not we're talking about here are we? The other tasks, like
> backporting changes for example, have been taken on by project teams.
> Even in your other email you mentioned that you've been doing backports
> and other tasks that you consider stable maint in a glance only context. That's
> something we changed in kilo which ttx referenced in [1] to enable that to
> happen, and it was the only way to scale things.
> 
> The discussion here is about the cross project effort around stable branches,
> which by design is a more limited scope now. Right now the cross project
> effort around stable branch policy is really 2 things (both of which ttx already
> mentioned):
> 
> 1. Keeping the gates working on the stable branches 2. Defining and
> enforcing stable branch policy.
> 
> The only lever on #2 is that the global stable-maint-core is the only group
> which has add permissions to the per project stable core groups. (also the
> stable branch policy wiki, but that rarely changes) We specifically shrunk it to
> these 2 things in. [1] Well, really 3 things there, but since we're not doing
> integrated stable point releases in the future its now only 2.
> 
> This is my whole argument that creating a new team doesn't do anything.
> Living under rel-mgt, some other project, or creating a new one isn't going to
> change the fact that cross-project stable maint is the same 2 tasks which
> basically nobody cares about, which TBH your email is just an indication of.
> Even if we wanted to grow to something beyond these 2 tasks they would
> still be the core of whatever it becomes and a lack of people caring about
> them will just block any potential scope growth.
> 
> Frankly, this is the problem with any ML or open discussion about stable
> branch maint. Everyone has an opinion about everything but lacks actual
> context or never steps up to work on the cross-project side of this. I don't
> think creating a new team that has no repos or other artifacts and therefore
> no stackalytics credit (and by extension corporate chest thumping) is
> magically going to create people actually working on this.
> 
> The logical follow on idea to the above is to create a stable branch policy doc
> repo and a project team to own that. But, I still don't think that solves the
> problem, especially since the real priority issue in this space is #1 which at
> most is a handful of us bothering to look at that, (well that and the stable
> policy barely ever changes) which honestly really isn't the majority of the
> stable-maint-core group.  We also have a similar problem with gate issues on
> master and it's mostly the same people looking at these things there too.
> 
> IMHO the only way for creating a separate project team to be useful is to re-
> centralize more responsibilities of stable maint, which is something I don't
> think we should do because we'll just hit the same scaling issues we had
> before kilo which prompted that change in the first place. It's the same
> problem every horizontal, cross project, or whatever you want to call it effort
> has with OpenStack's growth and why most have moved to the distributed
> self service models.
> 
> But, if the goal is just to not make Doug responsible for #2, which is
> something ttx was primarily doing before, then I guess it's fine but we should
> be honest about it and just make ttx the new team leader. :) I honestly don't
> think anything else discussed will change by explicitly making it a separate
> team.
> 
> -Matt Treinish
> 

Why not? I mean if there is interest to get stable separated from release management and having it's own team, why shouldn't we make it more effective? While I do agree that what you described is the current situation, I'd like to disagree it being the best way forward. If Thierry is willing to take the team and develop the stable efforts forward, that would be great.

What I was proposing in my first e-mail to the thread was to include the stable contributors (not just gatekeepers) to the team. Having those dedicated folks in the projects working on their project stable branches which would include them directly to the stable-maint team. Having the stable-maint-core being the core part of team still owning those <project>-stable-maint (core) groups. What I'm looking here is common grounds around stable maintenance combining the people who cares about the stable releases under same umbrella. Hopefully more working together, sharing processes and experiences about what works and what doesn't.

In a sense that would bring [1] back centralized, but instead of trying to find the people to do that, lets just include everyone who is doing that. What comes to the statistics, they obviously can be massaged to what direction ever. Still I wouldn't mind seeing the stable efforts being listed (if that's reason for organization X to put some effort to triage bugs against stable branches and do the backporting & reviews, gr8). Personally I just fail to see how this opportunity to improve things is excluded by the current situation.

The Stable Branch Policy, lets not put it in any repo. Doing that would give indication that we expect changes into it and I'd prefer that policy mostly staying stable as well.

I guess what I'm chasing here is that I'm not the only one frustrated around totally different experience about stable releases across the projects and their state. I'd be more than happy having some framework through which I can put my efforts in coming months trying to improve that. If that means that we declare it doomed after couple of cycles, so be it, but at least I'd be willing to take the risk hoping that something good comes out of it.

- Erno

> > >
> > > > >
> > > > > [1]
> > > > > http://lists.openstack.org/pipermail/openstack-dev/2015-November
> > > > > /078
> > > > > 281.html
> > > > >