<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 24, 2015 at 2:38 AM, Thierry Carrez <span dir="ltr"><<a href="mailto:thierry@openstack.org" target="_blank">thierry@openstack.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Joe Gordon wrote:<br>
> [...]<br>
<span class="">> I think a lot of the frustration with our current cadence comes out of<br>
> the big stop everything (development, planning etc.), and stabilize the<br>
> release process. Which in turn isn't really making things more stable.<br>
<br>
</span>I guess I have a different position. I think the release candidate<br>
period is the only thing that makes your code drops actually usable.<br>
It's the only moment in the cycle where integrators test. It's the only<br>
moment in the cycle where developers work on bugs they did not file<br>
themselves, but focus on a project-wide priority list of release<br>
blockers. If you remove that period, then nobody will ever work on<br>
release blockers that do not directly affect them. Shorten that period<br>
to one week, and no integrator will have the time to run a proper QA<br>
program to detect those release blockers.<br></blockquote><div><br></div><div>I still think the 3 week RC candidate cycle needs to happen, the difference is it would be done by stable maintenance. And I agree, the RC candidate period is one of the few moments where developers work on bugs they did not file themselves. So I am not sure how this would actually work. Perhaps the answer is we have deeper issues if we don't want to fix bugs until the last minute.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
I understand that from your developer perspective it's annoying to have<br>
to work on project-wide priorities rather than your own, and therefore<br>
you'd like to get rid of those -- but the resulting drop in quality is<br>
just something we can't afford.<br>
<span class=""><br>
> So I propose we keep the 6 month release cycle, but change the<br>
> development cycle from a 6 month one with 3 intermediate milestones to a<br>
> 6 week one with a milestone at the end of it.<br>
><br>
> What this actually means:<br>
><br>
</span>> * Stop approving blueprints for specific stable releases, instead just<br>
<span class="">> approve them and target them to milestones.<br>
</span>> o Milestones stop becoming Kilo-1, Kilo-2, Kilo-3 etc. and just<br>
> become 1,2,3,4,5,6,7,8,9 etc.<br>
> o If something misses what was previously known as Kilo-3 it has<br>
<span class="">> to wait a week for what for milestone 4.<br>
</span>> * Development focuses on milestones only. So 6 week cycle with say 1<br>
<span class="">> week of stabilization, finish things up before each milestone<br>
</span>> * Process of cutting a stable branch (planning, making the branch,<br>
<span class="">> doing release candidates, testing etc.) is done by a dedicated<br>
> stable branch team. And it should be done based on a specific milestone<br>
</span>> * Goal: Change the default development planning mode to ignore stable<br>
<span class="">> branches, and allow for us to think of things in terms of the number<br>
> of milestones needed, not will it make the stable branch or not<br>
<br>
</span>I don't think that would solve any of the issues you mentioned:<br>
> Current issues<br>
> * 3 weeks of feature freeze for all projects at the end of each cycle<br>
<span class="">> (3 out of the 26 feature blocked)<br>
<br>
</span>So you'll have 3 x 1 week of feature freeze for all projects, instead of<br>
1 x 3 weeks. That will be less efficient (integrators need a >1week<br>
feature freeze period to actually start testing a non-moving target),<br>
more painful (have to organize it 3 times instead of 1), and likely<br>
inefficient (takes generally more than one week to find critical bugs,<br>
develop the fix, and get it reviewed). And in the end, it's still 3 out<br>
of the 26 feature blocked.<br></blockquote><div><br></div><div>As said before, I don't envision integrator consuming every milestone. Just the standard 3 week RC cycle for stable branch candidates.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
> * 3 weeks of release candidates. Once a candidate is cut development<br>
<span class="">> is open for next release. While this is good in theory, not much<br>
> work actually starts on next release.<br>
<br>
</span>That is not really what I observe. People start landing their feature in<br>
the master branch starting the day after RC1. I actually observe the<br>
opposite: too many people switching to master development, and not<br>
enough people working on RC2+ bugs.<br></blockquote><div><br></div><div>Unfortunately I think we are both right. To many people move on and don't work on RC2 bugs, but development still slows down.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
> * some projects have non priority feature freezes and at Milestone 2<br>
<span class="">> (so 9 out of 26 weeks restricted in those projects)<br>
<br>
</span>That was their own choice. I for one was really surprised that they<br>
would freeze earlier -- I think 3 weeks is the right balance.<br>
<br>
> * vicious development circle<br>
> o vicious circle:<br>
> + big push right to land lots of features right before the release<br>
<br>
I think you'll have exactly the same push before the "stable" milestone<br>
(or the one that will be adopted by $DISTRO).<br></blockquote><div><br></div><div>I am hoping the push would be smaller, but I don't think we can remove it completely.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
>> + a lot of effort is spent getting the release ready<br>
>> + after the release people are a little burnt out and take it<br>
<span class="">>> easy until the next summit<br>
<br>
</span>Not convinced the burn out will be less significant with 4 releases<br>
instead of one every 6 months. Arguably it will be more distributed. But<br>
the main reason for the drop in activity between release and summit is<br>
that a lot of companies organize their team gatherings around that time<br>
(to prepare for summit).<br>
<br>
>> + Blueprints have to be re-discussed and re-approved for the<br>
>> next cycle<br>
>> + makes it hard to land blueprints early in the cycle casing<br>
<span class="">>> the bug rush at the end of the cycle, see step 1<br>
<br>
</span>Re-approving is each project choice. Personally I think revetting<br>
already-approved specs is a bit of madness.<br>
<br>
>> o Makes it hard to plan things across a release<br>
<br>
I don't see how the proposed change really addresses that. Do you plan<br>
to have 4 midcycle sprints ?<br></blockquote><div><br></div><div>No, instead of saying 'this has to wait until Lemming,' so wait at least a month. We can say ok, we expect this feature to take 4 milestones (and not try to map it to a stable branch).</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
>> o This actually destabilizes things right as we go into the<br>
<span class="">>> stabilization period (We actually have great data on this too)<br>
</span>>> o Means postponing blueprints that miss a deadline several months<br>
<br>
I think transforming 3 milestones + 1 release into 3 intermediary<br>
releases and a "stable" final release will exhibit *exactly* the same<br>
issues. </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<span class=""><font color="#888888"><br>
--<br>
Thierry Carrez (ttx)<br>
</font></span><div class=""><div class="h5"><br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div></div>