On Fri, Nov 5, 2021 at 10:39 AM Thierry Carrez <thierry@openstack.org> wrote:
>
> Hi everyone,
>
> The (long) document below reflects the current position of the release
> management team on a popular question: should the OpenStack release
> cadence be changed? Please note that we only address the release
> management / stable branch management facet of the problem. There are
> other dimensions to take into account (governance, feature deprecation,
> supported distros...) to get a complete view of the debate.
>
> Introduction
> ------------
>
> The subject of how often OpenStack should be released has been regularly
> debated in the OpenStack community. OpenStack started with a 3-month
> release cycle, then switched to 6-month release cycle starting with
> Diablo. It is often thought of a release management decision, but it is
> actually a much larger topic: a release cadence is a trade-off between
> pressure to release more often and pressure to release less often,
> coming in from a lot of different stakeholders. In OpenStack, it is
> ultimately a Technical Committee decision. But that decision is informed
> by the position of a number of stakeholders. This document gives
> historical context and describes the current release management team
> position.
>
> The current trade-off
> ---------------------
>
> The main pressure to release more often is to make features available to
> users faster. Developers get a faster feedback loop, hardware vendors
> ensure software is compatible with their latest products, and users get
> exciting new features. "Release early, release often" is a best practice
> in our industry -- we should generally aim at releasing as often as
> possible.
>
> But that is counterbalanced by pressure to release less often. From a
> development perspective, each release cycle comes with some process
> overhead. On the integrators side, a new release means packaging and
> validation work. On the users side, it means pressure to upgrade. To
> justify that cost, there needs to be enough user-visible benefit (like
> new features) in a given release.
>
> For the last 10 years for OpenStack, that balance has been around six
> months. Six months let us accumulate enough new development that it was
> worth upgrading to / integrating the new version, while giving enough
> time to actually do the work. It also aligned well with Foundation
> events cadence, allowing to synchronize in-person developer meetings
> date with start of cycles.
>
> What changed
> ------------
>
> The major recent change affecting this trade-off is that the pace of new
> development in OpenStack slowed down. The rhythm of changes was divided
> by 3 between 2015 and 2021, reflecting that OpenStack is now a mature
> and stable solution, where accessing the latest features is no longer a
> major driver. That reduces some of the pressure for releasing more
> often. At the same time, we have more users every day, with larger and
> larger deployments, and keeping those clusters constantly up to date is
> an operational challenge. That increases the pressure to release less
> often. In essence, OpenStack is becoming much more like a LTS
> distribution than a web browser -- something users like moving slow.
>
> Over the past years, project teams also increasingly decoupled
> individual components from the "coordinated release". More and more
> components opted for an independent or intermediary-released model,
> where they can put out releases in the middle of a cycle, making new
> features available to their users. This increasingly opens up the
> possibility of a longer "coordinated release" which would still allow
> development teams to follow "release early, release often" best
> practices. All that recent evolution means it is (again) time to
> reconsider if the 6-month cadence is what serves our community best, and
> in particular if a longer release cadence would not suit us better.
>
> The release management team position on the debate
> --------------------------------------------------
>
> While releasing less often would definitely reduce the load on the
> release management team, most of the team work being automated, we do
> not think it should be a major factor in motivating the decision. We
> should not adjust the cadence too often though, as there is a one-time
> cost in switching our processes. In terms of impact, we expect that a
> switch to a longer cycle will encourage more project teams to adopt a
> "with-intermediary" release model (rather than the traditional "with-rc"
> single release per cycle), which may lead to abandoning the latter,
> hence simplifying our processes. Longer cycles might also discourage
> people to commit to PTL or release liaison work. We'd probably need to
> manage expectations there, and encourage more frequent switches (or
> create alternate models).
>
> If the decision is made to switch to a longer cycle, the release
> management team recommends to switch to one year directly. That would
> avoid changing it again anytime soon, and synchronizing on a calendar
> year is much simpler to follow and communicate. We also recommend
> announcing the change well in advance. We currently have an opportunity
> of making the switch when we reach the end of the release naming
> alphabet, which would also greatly simplify the communications around
> the change.
>
> Finally, it is worth mentioning the impact on the stable branch work.
> Releasing less often would likely impact the number of stable branches
> that we keep on maintaining, so that we do not go too much in the past
> (and hit unmaintained distributions or long-gone dependencies). We
> currently maintain releases for 18 months before they switch to extended
> maintenance, which results in between 3 and 4 releases being maintained
> at the same time. We'd recommend switching to maintaining one-year
> releases for 24 months, which would result in between 2 and 3 releases
> being maintained at the same time. Such a change would lead to longer
> maintenance for our users while reducing backporting work for our
> developers.
>

Thanks for the write up Thierry.

I wonder what are the thoughts of the community of having LTS + normal releases
so that we can have the power of both?  I guess that is essentially what we have
with EM, but I guess we could introduce a way to ensure that operators can just
upgrade LTS to LTS.

It can complicate things a bit from a CI and project management side,
but I think it
could solve the problem for both sides that need want new features +
those who want
stability?

> --
> Thierry Carrez (ttx)
> On behalf of the OpenStack Release Management team
>


--
Mohammed Naser
VEXXHOST, Inc.