Open Stack

On Tue, Mar 3, 2015 at 9:45 AM, James Bottomley <
James.Bottomley at hansenpartnership.com> wrote:

> On Tue, 2015-02-24 at 12:05 +0100, Thierry Carrez wrote:
> > Daniel P. Berrange wrote:
> > > [...]
> > > The key observations
> > > ====================
> > >
> > > The first key observation from the schedule is that although we have
> > > a 6 month release cycle, we in fact make 4 releases in that six
> > > months because there are 3 milestones releases approx 6-7 weeks apart
> > > from each other, in addition to the final release. So one of the key
> > > burdens of a more frequent release cycle is already being felt, to
> > > some degree.
> > >
> > > The second observation is that thanks to the need to support a
> > > continuous deployment models, the GIT master branches are generally
> > > considered to be production ready at all times. The tree does not
> > > typically go through periods of major instability that can be seen
> > > in other projects, particular those which lack such comprehensive
> > > testing infrastructure.
> > >
> > > The third observation is that due to the relatively long cycle, and
> > > increasing amounts of process, the work accomplished during the
> > > cycles is becoming increasingly bursty. This is in turn causing
> > > unacceptably long delays for contributors when their work is unlucky
> > > enough to not get accepted during certain critical short windows of
> > > opportunity in the cycle.
> > >
> > > The first two observations strongly suggest that the choice of 6
> > > months as a cycle length is a fairly arbitrary decision that can be
> > > changed without unreasonable pain. The third observation suggests a
> > > much shorter cycle length would smooth out the bumps and lead to a
> > > more efficient & satisfying development process for all involved.
> >
> > I think you're judging the cycle from the perspective of developers
> > only. 6 months was not an arbitrary decision. Translations and
> > documentation teams basically need a month of feature/string freeze in
> > order to complete their work. Since we can't reasonably freeze one month
> > every 2 months, we picked 6 months.
>
> Actually, this is possible: look at Linux, it freezes for 10 weeks of a
> 12 month release cycle (or 6 weeks of an 8 week one).  More on this
> below.
>
> > It's also worth noting that we were on a 3-month cycle at the start of
> > OpenStack. That was dropped after a cataclysmic release that managed the
> > feat of (a) not having anything significant done, and (b) have out of
> > date documentation and translations.
> >
> > While I agree that the packagers and stable teams can opt to skip a
> > release, the docs, translations or security teams don't really have that
> > luxury... Please go beyond the developers needs and consider the needs
> > of the other teams.
> >
> > Random other comments below:
> >
> > > [...]
> > > Release schedule
> > > ----------------
> > >
> > > First the releases would probably be best attached to a set of
> > > pre-determined fixed dates that don't ever vary from year to year.
> > > eg releses happen Feb 1st, Apr 1st, Jun 1st, Aug 1st, Oct 1st, and
> > > Dec 1st. If a particular release slips, don't alter following release
> > > dates, just shorten the length of the dev cycle, so it becomes fully
> > > self-correcting. The even numbered months are suggested to avoid a
> > > release landing in xmas/new year :-)
> >
> > The Feb 1 release would probably be pretty empty :)
> >
> > > [...]
> > > Stable branches
> > > ---------------
> > >
> > > The consequences of a 2 month release cycle appear fairly severe for
> > > the stable branch maint teams at first sight. This is not, however,
> > > an insurmountable problem. The linux kernel shows an easy way forward
> > > with their approach of only maintaining stable branches for a subset
> > > of major releases, based around user / vendor demand. So it is still
> > > entirely conceivable that the stable team only provide stable branch
> > > releases for 2 out of the 6 yearly releases. ie no additional burden
> > > over what they face today. Of course they might decide they want to
> > > do more stable branches, but maintain each for a shorter time. So I
> > > could equally see them choosing todo 3 or 4 stable branches a year.
> > > Whatever is most effective for those involved and those consuming
> > > them is fine.
> >
> > Stable branches may have the luxury of skipping releases and designate a
> > "stable" one from time to time (I reject the Linux comparison because
> > the kernel is at a very different moment in software lifecycle). The
> > trick being, making one release "special" is sure to recreate the peak
> > issues you're trying to solve.
>
> I don't disagree with the observation about different points in the
> lifecycle, but perhaps it might be instructive to ask if the linux
> kernel ever had a period in its development history that looks somewhat
> like OpenStack does now.  I would claim it did: before 2.6, we had the
> odd/even develop/stabilise cycle.  The theory driving it was that we
> needed a time for everyone to develop then a time for everyone to help
> make stable.  You yourself said this in the other thread:
>
> > Joe Gordon wrote:
> > > [...]
> > > I think a lot of the frustration with our current cadence comes out of
> > > the big stop everything (development, planning etc.), and stabilize the
> > > release process. Which in turn isn't really making things more stable.
> >
> > I guess I have a different position. I think the release candidate
> > period is the only thing that makes your code drops actually usable.
> > It's the only moment in the cycle where integrators test. It's the
> > only
> > moment in the cycle where developers work on bugs they did not file
> > themselves, but focus on a project-wide priority list of release
> > blockers. If you remove that period, then nobody will ever work on
> > release blockers that do not directly affect them. Shorten that period
> > to one week, and no integrator will have the time to run a proper QA
> > program to detect those release blockers.
>
> This is a precise echo of the driver for the Linux kernel odd/even
> develop/stabilise cycle.  However, the develop/stabilise cycle lead to a
> huge amount of frustration from people who wanted to add new features
> while we were stabilising (and from distributions who needed a release
> while we were developing).
>
> In the beginning, for Linux Kernel: 0.9/1.0; 1.1/1.2; 1.3/2.0; 2.1/2.2
> all had relatively short cycles for develop/stabilise, but the
> frustrations in the system meant that by the time we got to 2.3/2.4 the
> cycle was out of control and 2.5 exploded for us and led to the adoption
> of the new model for 2.6 that you see today.
>
> The ultimate way we resolved the tensions was to acknowledge that
> developers will develop.  If they're interested, they'll help you
> stabilise, but if not they'll just back bite about their frustrations
> trying to get their next feature in.  Conversely, you have a whole
> ecosystem (the OpenStack distributions) interested in stability, so
> they'll help you stabilise. So the Linux Kernel now do an overlapping
> develop/release cycle, where it looks like we stabilise for n-2 weeks of
> the n week cycle, but in reality developers develop the N+1 release in
> that time-frame and the people interested in stability stabilise the
> release.  The result is that a lot of the developer frustration
> evaporated (they only get yelled at if they try to discuss patches with
> maintainers in the 2-week merge window period) and stability hasn't
> really suffered.
>
> This is a long winded way of saying that Daniel has the right idea (in
> my opinion) but it would depend on adopting different processes than
> today.
>

For fun, here is a really old thread, that tried to suggest a kernel-like
process:
"Nova subsystem branches and feature branches"
http://www.gossamer-threads.com/lists/openstack/dev/12708

-Angus


>
> James
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150303/50a87912/attachment.html>