[openstack-dev] [tc] [all] OpenStack moving both too fast and too slow at the same time

Zane Bitter zbitter at redhat.com
Fri May 5 00:09:35 UTC 2017

On 04/05/17 10:14, Thierry Carrez wrote:
> Chris Dent wrote:
>> On Wed, 3 May 2017, Drew Fisher wrote:
>>> "Most large customers move slowly and thus are running older versions,
>>> which are EOL upstream sometimes before they even deploy them."
>> Can someone with more of the history give more detail on where the
>> expectation arose that upstream ought to be responsible things like
>> long term support? I had always understood that such features were
>> part of the way in which the corporately avaialable products added
>> value?
> We started with no stable branches, we were just producing releases and
> ensuring that updates vaguely worked from N-1 to N. There were a lot of
> distributions, and they all maintained their own stable branches,
> handling backport of critical fixes. That is a pretty classic upstream /
> downstream model.
> Some of us (including me) spotted the obvious duplication of effort
> there, and encouraged distributions to share that stable branch
> maintenance work rather than duplicate it. Here the stable branches were
> born, mostly through a collaboration between Red Hat developers and
> Canonical developers. All was well. Nobody was saying LTS back then
> because OpenStack was barely usable so nobody wanted to stay on any
> given version for too long.

Heh, if you go back _that_ far then upgrades between versions basically 
weren't feasible, so everybody stayed on a given version for too long. 
It's true that nobody *wanted* to though :D

> Maintaining stable branches has a cost. Keeping the infrastructure that
> ensures that stable branches are actually working is a complex endeavor
> that requires people to constantly pay attention. As time passed, we saw
> the involvement of distro packagers become more limited. We therefore
> limited the number of stable branches (and the length of time we
> maintained them) to match the staffing of that team.

I wonder if this is one that needs revisiting. There was certainly a 
time when closing a branch came with a strong sense of relief that you 
could stop nursing the gate. I personally haven't felt that way in a 
couple of years, thanks to a lot of *very* hard work done by the folks 
looking after the gate to systematically solve a lot of those recurring 
issues (e.g. by introducing upper constraints). We're still assuming 
that stable branches are expensive, but what if they aren't any more?

> Fast-forward to
> today: the stable team is mostly one person, who is now out of his job
> and seeking employment.
> In parallel, OpenStack became more stable, so the demand for longer-term
> maintenance is stronger. People still expect "upstream" to provide it,
> not realizing upstream is made of people employed by various
> organizations, and that apparently their interest in funding work in
> that area is pretty dead.
> I agree that our current stable branch model is inappropriate:
> maintaining stable branches for one year only is a bit useless. But I
> only see two outcomes:
> 1/ The OpenStack community still thinks there is a lot of value in doing
> this work upstream, in which case organizations should invest resources
> in making that happen (starting with giving the Stable branch
> maintenance PTL a job), and then, yes, we should definitely consider
> things like LTS or longer periods of support for stable branches, to
> match the evolving usage of OpenStack.

Speaking as a downstream maintainer, it sucks that backports I'm still 
doing to, say, Liberty don't benefit anybody but Red Hat customers, 
because there's nowhere upstream that I can share them. I want everyone 
in the community to benefit. Even if I could only upload patches to 
Gerrit and not merge them, that would at least be something.

(In a related bugbear, whyyyyy must we delete the branch at EOL? This is 
pure evil for consumers of the code. It breaks existing git checkouts 
and thousands of web links in bug reports, review comments, IRC logs...)

> 2/ The OpenStack community thinks this is better handled downstream, and
> we should just get rid of them completely. This is a valid approach, and
> a lot of other open source communities just do that.

Maybe we need a 5th 'Open', because to me the idea that the software 
isn't so much 'released' as 'abandoned' is problematic in many of the 
same ways that Open Core and code dumps are.


> The current reality in terms of invested resources points to (2). I
> personally would prefer (1), because that lets us address security
> issues more efficiently and avoids duplicating effort downstream. But
> unfortunately I don't control where development resources are posted.

More information about the OpenStack-dev mailing list