[openstack-dev] [all] The future of the integrated release

Stefano Maffulli stefano at openstack.org
Thu Aug 7 17:38:45 UTC 2014

On 08/07/2014 01:49 AM, Thierry Carrez wrote:
> As an ex factory IT manager, I feel compelled to comment on that :)

Multidisciplinary training rules! As an architect with field experience
building roads, sidewalks, roofs, city planning (and training in lean
manufacturing and services) I think I can have a say ;)

> You're not really introducing a successful Kanban here, you're just
> clarifying that there should be a set number of workstations.

Right, and to clarify I'm really thinking kanban here, expanding on the
few lines Mikal used to explain the 'slots' concept.

> Our current system is like a gigantic open space with hundreds of
> half-finished pieces, and a dozen workers keep on going from one to
> another with no strong pattern. The proposed system is to limit the
> number of half-finished pieces fighting for the workers attention at any
> given time, by setting a clear number of workstations.

Correct, and I think we should add a limit to the amount of WIP, too. So
we have a visible limit to people, workstations and Work In Progress.
This way we can *immediately*, at any given time, identify problems.

> A true Kanban would be an interface between developers and reviewers,
> where reviewers define what type of change they have to review to
> complete production objectives, *and* developers would strive to produce
> enough to keep the kanban above the red line, but not too much (which
> would be piling up waste).

Exactly where I am aiming at: reducing waste, which we already have but
nobody (few, at different times) sees. By switching to a pull 'Just In
Time' mode we'd see waste accumulate much earlier than as we do now.

> Without that discipline, Kanbans are useless. Unless the developers
> adapt what they work on based on release objectives, you don't really
> reduce waste/inventory at all, it just piles up waiting for available
> "runway slots". As I said in my original email, the main issue here is
> the imbalance between too many people proposing changes and not enough
> people caring about the project itself enough to be trusted with core
> reviewers rights.

I agree with you. Right now we're accumulating waste in the form of code
proposals (raw pieces that need to be processed) but reviewers and core
reviewers' attention span is limited (the number of 'workstations' is
finite but we don't have such limit exposed) and nobody sees the
accumulation of backlog until it's very late, at the end of the release

A lot of the complaints I hear and worsening time to merge patches seems
to indicate that we're over capacity and we didn't notice.

> The only way to be truly pull-based is
> to define a set of production objectives and have those objectives
> trickle up to the developers so that they don't work on something else.

Yeah, don't we try to do that with blueprints/specs and priority? But we
don't set a limit, it's almost a 'free for all' send your patches in and
someone will evaluate them. Except there is a limit to what we can produce.

I think fundamentally we need to admit that there are 24 hours in a day
and that core reviewers have to sleep, sometimes. There is a finite
amount of patches that can be processed in a given time interval.

It's about finding a way to keep producing OpenStack at the highest
speed possible, keeping quality, listening to 'downstream' first.

> The solution is about setting release cycle goals and strongly
> communicating that everything out of those goals is clearly priority 2.

I don't think there is any 'proposal' just yet, only a half-baked idea
thrown out there by the nova team during a meeting and fluffed up by me
on the list. Still only a half-baked idea.

I realized this is a digression from the original thread though. I'll
talk to Russell and Nikola off-list (since they sent interesting
comments, too) and John and Dan to see if they're still interested in
formulating a comprehensive proposal.


Ask and answer questions on https://ask.openstack.org

More information about the OpenStack-dev mailing list