[all][tc] Relmgt team position on release cadence

Dmitry Tantsur dtantsur at redhat.com
Sat Nov 6 15:22:23 UTC 2021


On Fri, Nov 5, 2021 at 5:04 PM Mohammed Naser <mnaser at vexxhost.com> wrote:

> On Fri, Nov 5, 2021 at 10:39 AM Thierry Carrez <thierry at openstack.org>
> wrote:
> >
> > Hi everyone,
> >
> > The (long) document below reflects the current position of the release
> > management team on a popular question: should the OpenStack release
> > cadence be changed? Please note that we only address the release
> > management / stable branch management facet of the problem. There are
> > other dimensions to take into account (governance, feature deprecation,
> > supported distros...) to get a complete view of the debate.
> >
> > Introduction
> > ------------
> >
> > The subject of how often OpenStack should be released has been regularly
> > debated in the OpenStack community. OpenStack started with a 3-month
> > release cycle, then switched to 6-month release cycle starting with
> > Diablo. It is often thought of a release management decision, but it is
> > actually a much larger topic: a release cadence is a trade-off between
> > pressure to release more often and pressure to release less often,
> > coming in from a lot of different stakeholders. In OpenStack, it is
> > ultimately a Technical Committee decision. But that decision is informed
> > by the position of a number of stakeholders. This document gives
> > historical context and describes the current release management team
> > position.
> >
> > The current trade-off
> > ---------------------
> >
> > The main pressure to release more often is to make features available to
> > users faster. Developers get a faster feedback loop, hardware vendors
> > ensure software is compatible with their latest products, and users get
> > exciting new features. "Release early, release often" is a best practice
> > in our industry -- we should generally aim at releasing as often as
> > possible.
> >
> > But that is counterbalanced by pressure to release less often. From a
> > development perspective, each release cycle comes with some process
> > overhead. On the integrators side, a new release means packaging and
> > validation work. On the users side, it means pressure to upgrade. To
> > justify that cost, there needs to be enough user-visible benefit (like
> > new features) in a given release.
> >
> > For the last 10 years for OpenStack, that balance has been around six
> > months. Six months let us accumulate enough new development that it was
> > worth upgrading to / integrating the new version, while giving enough
> > time to actually do the work. It also aligned well with Foundation
> > events cadence, allowing to synchronize in-person developer meetings
> > date with start of cycles.
> >
> > What changed
> > ------------
> >
> > The major recent change affecting this trade-off is that the pace of new
> > development in OpenStack slowed down. The rhythm of changes was divided
> > by 3 between 2015 and 2021, reflecting that OpenStack is now a mature
> > and stable solution, where accessing the latest features is no longer a
> > major driver. That reduces some of the pressure for releasing more
> > often. At the same time, we have more users every day, with larger and
> > larger deployments, and keeping those clusters constantly up to date is
> > an operational challenge. That increases the pressure to release less
> > often. In essence, OpenStack is becoming much more like a LTS
> > distribution than a web browser -- something users like moving slow.
> >
> > Over the past years, project teams also increasingly decoupled
> > individual components from the "coordinated release". More and more
> > components opted for an independent or intermediary-released model,
> > where they can put out releases in the middle of a cycle, making new
> > features available to their users. This increasingly opens up the
> > possibility of a longer "coordinated release" which would still allow
> > development teams to follow "release early, release often" best
> > practices. All that recent evolution means it is (again) time to
> > reconsider if the 6-month cadence is what serves our community best, and
> > in particular if a longer release cadence would not suit us better.
> >
> > The release management team position on the debate
> > --------------------------------------------------
> >
> > While releasing less often would definitely reduce the load on the
> > release management team, most of the team work being automated, we do
> > not think it should be a major factor in motivating the decision. We
> > should not adjust the cadence too often though, as there is a one-time
> > cost in switching our processes. In terms of impact, we expect that a
> > switch to a longer cycle will encourage more project teams to adopt a
> > "with-intermediary" release model (rather than the traditional "with-rc"
> > single release per cycle), which may lead to abandoning the latter,
> > hence simplifying our processes. Longer cycles might also discourage
> > people to commit to PTL or release liaison work. We'd probably need to
> > manage expectations there, and encourage more frequent switches (or
> > create alternate models).
> >
> > If the decision is made to switch to a longer cycle, the release
> > management team recommends to switch to one year directly. That would
> > avoid changing it again anytime soon, and synchronizing on a calendar
> > year is much simpler to follow and communicate. We also recommend
> > announcing the change well in advance. We currently have an opportunity
> > of making the switch when we reach the end of the release naming
> > alphabet, which would also greatly simplify the communications around
> > the change.
> >
> > Finally, it is worth mentioning the impact on the stable branch work.
> > Releasing less often would likely impact the number of stable branches
> > that we keep on maintaining, so that we do not go too much in the past
> > (and hit unmaintained distributions or long-gone dependencies). We
> > currently maintain releases for 18 months before they switch to extended
> > maintenance, which results in between 3 and 4 releases being maintained
> > at the same time. We'd recommend switching to maintaining one-year
> > releases for 24 months, which would result in between 2 and 3 releases
> > being maintained at the same time. Such a change would lead to longer
> > maintenance for our users while reducing backporting work for our
> > developers.
> >
>
> Thanks for the write up Thierry.
>

++ very well written!


>
> I wonder what are the thoughts of the community of having LTS + normal
> releases
> so that we can have the power of both?  I guess that is essentially what
> we have
> with EM, but I guess we could introduce a way to ensure that operators can
> just
> upgrade LTS to LTS.
>

This is basically what Ironic does: we release with the rest of OpenStack,
but we also do 2 more releases per cycle with their own bugfix/X.Y branches.

Dmitry


>
> It can complicate things a bit from a CI and project management side,
> but I think it
> could solve the problem for both sides that need want new features +
> those who want
> stability?
>
> > --
> > Thierry Carrez (ttx)
> > On behalf of the OpenStack Release Management team
> >
>
>
> --
> Mohammed Naser
> VEXXHOST, Inc.
>
>

-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael
O'Neill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20211106/4cc4d53b/attachment.htm>


More information about the openstack-discuss mailing list