Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Daniel,

thanks for a clear write-up of the matter and food for thought.

I think the idea of having more smooth development mode that would not
make people to wait for 6+ months to release a new feature is great.

It's insane to expect that feature priorities won't ever slightly
shift in the next 6 months. Having a limited list of features targeted
for the next 6 months is prone to mistakes, since people behind some
of approved features may need to postpone the effort for any type of
reasons (personal, job rush, bad resource allocation, ...), and we
then end up with approved specs with no actual code drops, using
review 'slots' that would better be spent for other features that were
not that lucky to get a rubber stamp during spec phase. Prior resource
allocation would probably work somehow if we were working for the same
company that would define priorities to us, but it's not the case.

Anecdotally, in neutron, we have two Essential blueprints for Kilo,
and there are no code drops or patches in review for any of those, so
I would expect them to fail to merge. At the same time, I will need to
wait for the end of Kilo to consider adding support for guru reports
to the project. Or in oslo world, I will need to wait for Liberty to
introduce features in oslo.policy that are needed by neutron to switch
to it, etc.

Another problem is that currently milestones are used merely for
targeting bugs and features, but no one really cares about whether the
actual milestone shipment actually works. Again, a story from neutron
world: Kilo-1 was shipped in the middle of advanced services split,
with some essential patches around distutils setup missing (no proper
migration plan applied, conflicting config files in neutron and *aas
repos, etc.)

So I'm all for reforms around processes we apply.

That said, I don't believe the real problem here is that we don't
generate project tarballs frequently enough.

Major problems I see as critical to tackle in our dev process are:

- - enforced spec/dev mode. Solution: allow to propose (and approve) a
reasonable spec at any moment in the cycle; allow code drops for
approved specs at any moment in the cycle (except pre-release
stabilization time); stop targeting specs: if it's sane, it's probably
sane N+2 cycle from now too.

- - core team rubber stamping a random set of specs and putting -2 on
all other specs due to "project priorities". Solution: stop pretending
core team (or core drivers) can reasonably estimate review and coding
resources for the next cycle. Instead allows community to decide
what's worth the effort by approving all technically reasonable specs
and allowing everyone to invest time and effort in specs (s)he seems
worth it.

- - no due stabilization process before dev milestones. Solution:
introduce one in your team workflow, be more strict in what goes in
during pre milestone stabilization time.

If all above is properly applied, we would get into position similar
to your proposal. The difference though is that upstream project would
not call milestone tarballs 'Releases'. Still, if there are brave
vendors to ship milestones, fine, they would have the same freedom as
in your proposal.

Note: all the steps mentioned above can be applied on *per team* basis
without breaking existing release workflow.

Some more comments from stable-maint/distribution side below.

On 02/24/2015 10:53 AM, Daniel P. Berrange wrote:
[...skip...]
> The modest proposal ===================
[...skip...]
> 
> Stable branches ---------------
> 
> The consequences of a 2 month release cycle appear fairly severe
> for the stable branch maint teams at first sight. This is not,
> however, an insurmountable problem. The linux kernel shows an easy
> way forward with their approach of only maintaining stable branches
> for a subset of major releases, based around user / vendor demand.
> So it is still entirely conceivable that the stable team only
> provide stable branch releases for 2 out of the 6 yearly releases.
> ie no additional burden over what they face today. Of course they
> might decide they want to do more stable branches, but maintain
> each for a shorter time. So I could equally see them choosing todo
> 3 or 4 stable branches a year. Whatever is most effective for those
> involved and those consuming them is fine.
> 

Since it's not only stable branches that are affected (translators,
documentation writers, VMT were already mentioned), those affected
will probably need to come up with some synchronized decision.

Let's say we still decide to support two out of six releases (same
scheme as is now). In that case no process that we usually attach to
releases will be running after release dates. This makes me wonder how
it's different from milestones we already have.

Do you think any downstream vendors will actually ship and support
upstream releases that upstream drops any guarantees for (no VMT, no
stable branches, no gate runs, ...) right after the release date? I
doubt there will be significant interest in it.

> Downstream vendors ------------------
> 
> Downstream product vendors would face a similar decision about how 
> they wished to produce. Some OS vendors may choose to ship all
> stable releases in their distros as they become available, others
> may choose to focus on a subset of the releases. In some cases more
> frequent release may actually simplify life for the vendors. For
> example, if there is a feature they need which doesn't make the cut
> for a release today, they currently have the choice of waiting 6
> months for the next release, or carrying non-upstream patches, or
> backporting patches from master. With a 2 month release cycle, it
> is more practical for vendors to simply wait 2 months for the next
> release to ship. So this would probably reduce the amount of patch 
> backporting vendors have todo in general, giving them more time to 
> focus on useful upstream work.
> 
> Design summit -------------
> 
> Another consequence of 2 month release cycles would be that the 
> design summits become entirely detached from the release cycles,
> as they'd be on different cycle lengths. So instead of having a
> focus on release planning, design summits would likely become a
> forum for general ongoing collaboration and discussions. Probably a
> half & half merge between the current design summit format and a
> more traditional open source conference. This also ought to result
> in less pressure for all contributors to attend all summits, and so
> be time efficient travelwise and beneficial in cost reduction for 
> vendors.
> 
> More interestingly, is that by detaching them from release cycles, 
> the summit organizers could have more flexibility with scheduling 
> the summits. They could be placed on the dates that are most
> suitable for desired travel locations, rather than dates suited to
> release schedules. Likewise there is more freedom if there was ever
> a wish to have design summits and the main track conference
> separated.
> 
> Election process ----------------
> 
> I wouldn't suggest any changes to project elections or governance. 
> It is fine to have elections every 6 months. There is no need for 
> that to be tied to release cycle length in any way, particularly as
> people often serve for more than one release already.
> 
> Release naming --------------
> 
> Last but not least, there would be some questions around what this 
> might mean for release naming. There are many options and I don't 
> intend to suggest a particular one is best, just highlight some 
> possibilities that could be considered.
> 
> The easiest is to just drop the idea of release names entirely and 
> have everyone refer to the release versions instead. Another
> option is to give each 2 month release its own name, so we'd have
> to come up with 3x as many names during the year. The person
> running the release naming elections will probably have a heart
> attack at this idea :-) Another option is to have names that span a
> couple of releases, perhaps all releases for an entire year get a
> given name.
> 
> Upgrades & deprecation ----------------------
> 
> It is common right now for projects to say upgrades are only 
> supported between releases N-1 and N. ie to go from Icehouse to
> Kilo, you need to first deploy Juno. This is passable when you're
> talking 6 month gaps between cycles, but when there are 2 month
> gaps it is not reasonable to expect everyone to be moving fast
> enough to keep up with every release. If an organization's
> beurocracy means they can't deploy more often than every 12 months,
> forcing them to deploy the 5 intermediate releases to run upgrade
> scripts is quite unpleasant. We would likely have to look at
> officially allowing upgrades between any (N-3, N-2, N-1) to N. From
> a database POV this should not be hard, since the DB migration
> scripts don't have any built in assumptions about this. Similarly
> the versioned objects used by Nova are quite flexible in this
> regard too, as long as the compat code isn't deleted too soon.

Let's say horizontal teams decided to stick to 2 supported releases
per year scheme (the current one). What would grenade jobs for stable
branches (and master) test in this case?

> 
> Deprecation warnings would need similar consideration. It would not
> be sufficient to deprecate in one release and delete in the next.
> We'd likely want to say that depecations last for a given time
> period rather than number of releases, eg 6 months. This could be
> easily handled by simply including the date of initial deprecation
> in the deprecation message. It would thus be clear when the feature
> will be removed to all involved.
> 
> Why do releases at all ? ------------------------
> 
> Some people might wonder why we should do releases at all. We 
> already aim to support a continuous deployment model for users as
> an option. Continuous deployment, however, is quite a conceptual
> jump for many users and organizations in particular. Their internal
> software approval processes and bureaucracy can make this
> impractical to even consider in the forseeable future. So if
> openstack did not do releases, then many downstream vendors would
> still end up taking git snapshots and forming periodic releases
> around them. The unsatisfactory result would be a balkanization of
> OpenStack where each vendor duplicated the others work with their
> own release cycles in slightly different ways. To some extent this
> happens already as vendors backport features, due to inability to
> wait 6 months for next major release. A two month release cycle
> though would likely reduce the amount of difference between what
> each vendor ships which would be a good thing in general. Releases
> also serve as a useful hook for marketing activities by the project
> and vendors alike, giving a good opportunity to promote the new
> body of features now available to users. It might be difficult to
> drum up media enthusiasm to promote arbitrary git snapshots, but
> then I'm no marketing expert.
> 

I think it's still easier for people to work in release/stability mode
for some predefined time than "forever".

> 
> 
> 
> Regards, Daniel
> 
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2014-September/044872.html
>
> 
[2] http://www.openstack.org/projects/openstack-faq/
> [3] https://wiki.openstack.org/wiki/Kilo_Release_Schedule [4]
> http://lists.openstack.org/pipermail/openstack-dev/2014-October/047922.html
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEbBAEBAgAGBQJU9IheAAoJEC5aWaUY1u57/M4H912jtcA8IP424DZnkDVm/6dF
uXl02JqeUeFvne+M9/JoRIO+GwuY05kfmwFAXY5RNX5PmW1MA+eLpirKI5waMjzJ
6zAwViir2sLtrXjpHNKkUrQxzURWN72qvnU+idCJlExU4ZXCkKcTuGo0YC6j46WH
mLuRheb3wRqmkMrZ7nlShaAiSkLBYpslP+NuKH4JuWzyAZqrQsTwcu9duOD9QiTR
294ojzqihCth7npMeXcdMtdkl+GoxP494ojouWqVTAYCpWDfTiZw3/Slb0VTYVIe
uIS1gN2Hmur1vTbkM4kgvU3D9Uf+uLS9vH9GDg+zyDsjQQm/6RToCOTFiAYp8w==
=nhsu
-----END PGP SIGNATURE-----