Open Stack

On 24 February 2015 at 13:14, Joe Gordon <joe.gordon0 at gmail.com> wrote:
> Was:
> http://lists.openstack.org/pipermail/openstack-dev/2015-February/057578.html
>
> There has been frustration with our current 6 month development cadence.
> This is an attempt to explain those frustrations and propose a very rough
> outline of a possible alternative.

Yup, I'm in the frustrated bucket - see the prior discussions on this
list about it :)

...> vicious circle:
>
> big push right to land lots of features right before the release
> a lot of effort is spent getting the release ready
> after the release people are a little burnt out and take it easy until the
> next summit
> Blueprints have to be re-discussed and re-approved for the next cycle

Re-assessing what we're doing is I think broadly useful (no point
doing things that are no longer relevant), but this particular thihng:

> makes it hard to land blueprints early in the cycle casing the bug rush at
> the end of the cycle, see step 1

Is indeed disruptive.


> Makes it hard to plan things across a release
> This actually destabilizes things right as we go into the stabilization
> period (We actually have great data on this too)
> Means postponing blueprints that miss a deadline several months

Related / variation on a theme - its hard to tackle any multi-cycle
effort today, because cycles.

>
> Requirements for a new model
>
> Keep 6 month release cadence. Not everyone is willing to deploy from trunk

Also what does it really mean ? Do you mean the 6 month design&do
cycle, or that we'll cut a release at least once every 6 months on any
project with changes?
I *think* you mean having a release at least once every 6 months. That
doesn't imply a 6 month cadence though. From a lean perspective our
cycle time determines our agility: having e.g. monthly cycles would be
a lot better IMO than 6 monthly cycles. Have a 6 monthly summit for
the benefits it brings, but have the actual development cadence be
4-weekly (or faster if we can).

> Keep stable branches for at least 6 months. In order to test upgrades from
> the last stable branch, we need a stable branch to test against

I think this assumes the answer. "Be able to reconstruct a working
OpenStack from 6 months ago" doesn't imply a stable branch at all
AFAICT. We do need an answer, and a branch of some form is some way to
do that.

> Keep supporting continuous deployment. Some people really want to deploy
> from trunk

AIUI Rackspace are approximately 2 weeks delayed, and I believe HP
public cloud are similar. If we make the gap between releases low
enough, I think we may well deliver enough agility for those
organisations: we should ask them in detail.


I'd like to offer these somewhat broader requirements instead (and we
can narrow scope down once we've some feeling for folks desires):

- have an artifact that RedHat, Ubuntu, Suse and so on can include in
their distributions. At most 6 months old when they cut their release.
(no worse than the status quo).

- have an artifact that large deployers such as RackSpace, HP public
cloud, Dreamhost etc can update in a stable graceful fashion (no worse
than the status quo).

- preserve stability and quality of releases


> What We can drop
...
> I think a lot of the frustration with our current cadence comes out of the
> big stop everything (development, planning etc.), and stabilize the release
> process. Which in turn isn't really making things more stable. So I propose
> we keep the 6 month release cycle, but change the development cycle from a 6
> month one with 3 intermediate milestones to a 6 week one with a milestone at
> the end of it.

I like that. but lets use the term release rather than milestone.

> What this actually means:
>
> Stop approving blueprints for specific stable releases, instead just approve
> them and target them to milestones.
>
> Milestones stop becoming Kilo-1, Kilo-2, Kilo-3 etc. and just become
> 1,2,3,4,5,6,7,8,9 etc.
> If something misses what was previously known as Kilo-3 it has to wait a
> week for what for milestone 4.
>
> Development focuses on milestones only. So 6 week cycle with say 1 week of
> stabilization, finish things up before each milestone
> Process of cutting a stable branch (planning, making the branch, doing
> release candidates, testing etc.) is done by a dedicated stable branch team.
> And it should be done based on a specific milestone
> Goal: Change the default development planning mode to ignore stable
> branches, and allow for us to think of things in terms of the number of
> milestones needed, not will it make the stable branch or not

+1 to that too

> Gotchas, questions etc:
>
> Some developers will still care about getting a feature into a specific
> stable release, so we may still get a small rush for the milestone before
> each stable branch

Lets wait to hear from them about why before we worry. Specifically if
there is a release 6 weeks later the only difference is if there will
be a 'stable branch' associated with the release or not. Since we
probably would be concerned about the overhead of maintaining a branch
for every 6 weekly release. But perhaps if we do only truely critical
fixes for each release that might be ok.

> Requires significantly more work from the stable maintenance team
> Naming the stable branches becomes less fun, as we refer to the stable
> branches less
> Since planning is continuous 6 month cycle for summits becomes a little
> strange. This is still an open ended question to me.

I think the summits with their 400 folk, 5 speaking pathology need a
rework anyway. We need to do the things that we can't do remotely
there: high bandwidth working groups seeking consensus on gnarly
problems, establish the real world relationships that let folk trust
each other and work together remotely, get devs working on thing X
time talking to deployers and operators about their needs with X in
breakout sessions etc.

-Rob




-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud