Open Stack

On Tue, Feb 24, 2015 at 2:38 AM, Thierry Carrez <thierry at openstack.org>
wrote:

> Joe Gordon wrote:
> > [...]
> > I think a lot of the frustration with our current cadence comes out of
> > the big stop everything (development, planning etc.), and stabilize the
> > release process. Which in turn isn't really making things more stable.
>
> I guess I have a different position. I think the release candidate
> period is the only thing that makes your code drops actually usable.
> It's the only moment in the cycle where integrators test. It's the only
> moment in the cycle where developers work on bugs they did not file
> themselves, but focus on a project-wide priority list of release
> blockers. If you remove that period, then nobody will ever work on
> release blockers that do not directly affect them. Shorten that period
> to one week, and no integrator will have the time to run a proper QA
> program to detect those release blockers.
>

I still think the 3 week RC candidate cycle needs to happen, the difference
is it would be done by stable maintenance. And I agree, the RC candidate
period is one of the few moments where developers work on bugs they did not
file themselves. So I am not sure how this would actually work.  Perhaps
the answer is we have deeper issues if we don't want to fix bugs until the
last minute.



>
> I understand that from your developer perspective it's annoying to have
> to work on project-wide priorities rather than your own, and therefore
> you'd like to get rid of those -- but the resulting drop in quality is
> just something we can't afford.
>
> > So I propose we keep the 6 month release cycle, but change the
> > development cycle from a 6 month one with 3 intermediate milestones to a
> > 6 week one with a milestone at the end of it.
> >
> > What this actually means:
> >
> >   * Stop approving blueprints for specific stable releases, instead just
> >     approve them and target them to milestones.
> >       o Milestones stop becoming Kilo-1, Kilo-2, Kilo-3 etc. and just
> >         become 1,2,3,4,5,6,7,8,9 etc.
> >       o If something misses what was previously known as Kilo-3 it has
> >         to wait a week for what for milestone 4.
> >   * Development focuses on milestones only. So 6 week cycle with say 1
> >     week of stabilization, finish things up before each milestone
> >   * Process of cutting a stable branch (planning, making the branch,
> >     doing release candidates, testing etc.) is done by a dedicated
> >     stable branch team. And it should be done based on a specific
> milestone
> >   * Goal: Change the default development planning mode to ignore stable
> >     branches, and allow for us to think of things in terms of the number
> >     of milestones needed, not will it make the stable branch or not
>
> I don't think that would solve any of the issues you mentioned:
> > Current issues
> >   * 3 weeks of feature freeze for all projects at the end of each cycle
> >     (3 out of the 26 feature blocked)
>
> So you'll have 3 x 1 week of feature freeze for all projects, instead of
> 1 x 3 weeks. That will be less efficient (integrators need a >1week
> feature freeze period to actually start testing a non-moving target),
> more painful (have to organize it 3 times instead of 1), and likely
> inefficient (takes generally more than one week to find critical bugs,
> develop the fix, and get it reviewed). And in the end, it's still 3 out
> of the 26 feature blocked.
>

As said before, I don't envision integrator consuming every milestone. Just
the standard 3 week RC cycle for stable branch candidates.


>
> >   * 3 weeks of release candidates. Once a candidate is cut development
> >     is open for next release. While this is good in theory, not much
> >     work actually starts on next release.
>
> That is not really what I observe. People start landing their feature in
> the master branch starting the day after RC1. I actually observe the
> opposite: too many people switching to master development, and not
> enough people working on RC2+ bugs.
>

Unfortunately I think we are both right. To many people move on and don't
work on RC2 bugs, but development still slows down.


>
> >   * some projects have non priority feature freezes and at Milestone 2
> >     (so 9 out of 26 weeks restricted in those projects)
>
> That was their own choice. I for one was really surprised that they
> would freeze earlier -- I think 3 weeks is the right balance.
>
> >   * vicious development circle
> >       o vicious circle:
> >           + big push right to land lots of features right before the
> release
>
> I think you'll have exactly the same push before the "stable" milestone
> (or the one that will be adopted by $DISTRO).
>

I am hoping the push would be smaller, but I don't think we can remove it
completely.


>
> >>           + a lot of effort is spent getting the release ready
> >>           + after the release people are a little burnt out and take it
> >>             easy until the next summit
>
> Not convinced the burn out will be less significant with 4 releases
> instead of one every 6 months. Arguably it will be more distributed. But
> the main reason for the drop in activity between release and summit is
> that a lot of companies organize their team gatherings around that time
> (to prepare for summit).
>
> >>           + Blueprints have to be re-discussed and re-approved for the
> >>             next cycle
> >>           + makes it hard to land blueprints early in the cycle casing
> >>             the bug rush at the end of the cycle, see step 1
>
> Re-approving is each project choice. Personally I think revetting
> already-approved specs is a bit of madness.
>
> >>       o Makes it hard to plan things across a release
>
> I don't see how the proposed change really addresses that. Do you plan
> to have 4 midcycle sprints ?
>

No, instead of saying 'this has to wait until Lemming,' so wait at least a
month. We can say ok, we expect this feature to take 4 milestones (and not
try to map it to a stable branch).

>
> >>       o This actually destabilizes things right as we go into the
> >>         stabilization period (We actually have great data on this too)
> >>       o Means postponing blueprints that miss a deadline several months
>
> I think transforming 3 milestones + 1 release into 3 intermediary
> releases and a "stable" final release will exhibit *exactly* the same
> issues.


> --
> Thierry Carrez (ttx)
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150224/1ca78769/attachment.html>