Open Stack

Excerpts from Markus Zoeller's message of 2016-01-21 18:37:00 +0100:
> Flavio Percoco <flavio at redhat.com> wrote on 01/21/2016 09:13:02 AM:
> 
> > From: Flavio Percoco <flavio at redhat.com>
> > To: "Daniel P. Berrange" <berrange at redhat.com>
> > Cc: "OpenStack Development Mailing List \(not for usage questions\)" 
> > <openstack-dev at lists.openstack.org>
> > Date: 01/21/2016 01:47 PM
> > Subject: Re: [openstack-dev] [all][tc] Stabilization cycles: 
> > Elaborating on the idea to move it forward
> > 
> > On 21/01/16 11:22 +0000, Daniel P. Berrange wrote:
> > >On Wed, Jan 20, 2016 at 01:23:02PM -0430, Flavio Percoco wrote:
> > >> Greetings,
> > >>
> > >> At the Tokyo summit, we discussed OpenStack's development themes in a
> > >> cross-project session. In this session a group of folks started 
> > discussing what
> > >> topics the overall community could focus on as a shared effort. One 
> of the
> > >> things that was raised during this session is the need of having 
> cycles to
> > >> stabilize projects. This was brought up by Robert Collins again in 
> > a meeting[0]
> > >> the TC had right after the summit and no much has been done ever 
> since.
> > >>
> > >> Now, "stabilization Cycles" are easy to dream about but really hardto 
> do and
> > >> enforce. Nonetheless, they are still worth a try or, at the very 
> least, a
> > >> thought. I'll try to go through some of the issues and benefits a 
> > stabilization
> > >> cycle could bring but bear in mind that the lists below are not 
> > exhaustive. In
> > >> fact, I'd love for other folks to chime in and help building a case
> > in favor or
> > >> against this.
> > >>
> > >> Negative(?) effects
> > >> ===================
> > >>
> > >> - Project won't get new features for a period of time Economic impact 
> on
> > >>  developers(?)
> > >> - It was mentioned that some folks receive bonuses for landed 
> features
> > >> - Economic impact on companies/market because no new features were 
> added (?)
> > >> - (?)
> > >
> > >It will push more development into non-upstream vendor private
> > >branches.
> > >
> > >>
> > >> Positive effects
> > >> ================
> > >>
> > >> - Focus on bug fixing
> > >> - Reduce review backlog
> > >> - Refactor *existing* code/features with cleanups
> > >> - Focus on multi-cycle features (if any) and complete those
> > >> - (?)
> > >
> > >I don't think the idea of stabalization cycles would really have
> > >such a positive effect, certainly not while our release cycle is
> > >6 months in length.
> > >
> > >If you say the next cycle is primarily stabalization, then what
> > >you are in effect saying is that people have to wait 12 months
> > >for their desired new feature.  In the fast moving world of
> > >cloud, I don't think that is a very credible approach. Even
> > >with our current workflow, where we selectively approve features
> > >for cycles, we have this impact of forcing people to wait 12
> > >months, or more, for their features.
> > 
> > ++
> > 
> > This is one of the main concerns and perhaps the reason why I don't 
> think it
> > should be all-or-nothing. It should be perfectly fine for teams to have
> > stabilization milestones, FWIW.
> > 
> > >In the non-stabalization cycle, we're not going to be able to
> > >merge a larger number of features than we already do today.
> > >So in effect we'll have 2 cycles worth of features being
> > >proposed for 1 cycle. When we inevitably reject moany of
> > >those features they'll have to wait for the next non-stabalization
> > >cycle, which means 18-24 months delay.
> > >
> > >Of course in reality this kind of delay won't happen. What will
> > >instead happen is that various vendors will get pressure from
> > >their customers/partners and their local branches of openstack
> > >packages will fork & diverge even further from upstream than
> > >they already do today.
> > >
> > >So while upstream branch will be "stabalized", most users will
> > >probably get a *less* stable release because they'll be using
> > >a branch from vendors with a tonne of non-upstream stuff added.
> > >
> > 
> > I would expect these vendors to (slowly?) push their changes 
> upstream.It'd take
> > time but it should certainly happen.
> > 
> > >In addition having a stablization cycle will give the impression
> > >that the following cycle is a non-stable one and likely cause
> > >more distruption by pushing lots of features in at one time.
> > >Instead of having a master branch which has an approximately
> > >constant level of stabalization, you'll create a situation
> > >where it fluctuates significantly, which is clearly worse for
> > >people doing continuous deployment.
> > >
> > >I think it is important to have the mindset that master should
> > >*always* be considered stable - we already have this in general
> > >and it is one of the success points of openstack's development
> > >model IMHO. The idea of stabalization cycles is a step backwards
> > 
> > Perhaps, it is being presented the wrong way. I guess the main point 
> here is how
> > ca we communicate that we'd like to take some time to clean-up the mess 
> we have
> > in some projects. How can projects ask their team to put more efforts on
> > tackling technical debt rather than pushing the new sexy thing?
> > 
> > I could consider Mitaka as a stabilization cycle for Glance (except for 
> the
> > upload path refactor spec). The team has spent quite some time on 
> working out a
> > way to improve that workflow. Few other specs have been implemented but 
> nothing
> > major, TBH (talking about Glance here, not the other components).
> > 
> > What I mean is, that I don't consider a stabilization cycle a full 
> heads-down on
> > bug fixing cyle but rather a cycle where no major features are approved. 
> What
> > unfortunatelly happens when these kind of cycles are announced or 
> planned is
> > that contributions vanish and they are routed to places where new 
> features land.
> > That should perhaps be an indicator of how good/bad these cycles are. 
> *shurgs*
> > 
> > >I still believe that if you want to improve stabality of the
> > >codebase, we'd be better off moving to a shorter development
> > >cycle. Even the 6 month cycle we have today is quite "lumpy"
> > >in terms of what kind of work happens from month to month. If
> > >we moved to a 2 month cycle, I think it would relieve pressure
> > >to push in features quickly before freeze, because people would
> > >know they'd have another opportunity very soon, instead of having
> > >to wait 6+ months. I've previously suggested that here:
> > >
> > >  
> http://lists.openstack.org/pipermail/openstack-dev/2015-February/057614.html
> 
> > >
> > 
> > Whether we move to shorter cycles or not, I still think there's a way we 
> can do
> > this now. Again, I don't believe these cycles should be 
> all-or-nothingand teams
> > should feel free to dedicate as much time to this as they want (and 
> > some do already).
> > 
> > Flavio
> > 
> > >Regards,
> > >Daniel
> 
> I try to handle in one post the different aspects which came up so far:
> 
> wrt dedicated stabilization cycles|milestones:
> 
>     Piled up (=older) bugs are harder to solve than fresh ones.
>     I've seen next to no bug report in Nova which has all the
>     necessary data to do a proper analysis. There are usually 
>     1-3 requests to the bug reporter necessary to get enough data.
>     This makes me believe that stabilization should be a continuous 
>     effort.

I agree. I think the intent here is not so much fixing lots of
individual bugs, but fixing those fundamental things that may take
a significant amount of thought and implementation time. I'm thinking
of things like performance issues, race conditions, etc. where just
reproducing the bug reliably is complex.

> 
> wrt cycle length:
> 
>     To get things merged in a specific cycle is indeed a big thing
>     for my employer (at least the parts I directly interact with).
>     A lot of effort goes into coordinating internal plans with the
>     OpenStack cycles. Decreasing the length of a cycle (2-4 months)
>     could make things a bit more relaxed.

All projects have the option of choosing a release model that allows
for multiple releases per 6 month cycle, while still maintaining a
single stable release after the cycle.

> 
> wrt "just fixing bugs":
> 
>     User experience is not only based on shiny features. I assume
>     that a fixed bug isn't a big differentiator for a company, which
>     makes that an unattractive task for them. The only possible 
>     motivator I can think of is prestige. A bullet point on a company 
>     slide that says "we solved 25% of the open bugs" could be a thing.
>     Having those "metrics" in the spotlight is maybe a thing?
>  
> 
> Regards, Markus Zoeller (markus_z)
>