[openstack-dev] Continuous deployment - significant process change

Sullivan, Jon Paul JonPaul.Sullivan at hp.com
Mon Apr 29 22:39:03 UTC 2013

> -----Original Message-----
> From: Russell Bryant [mailto:rbryant at redhat.com]
> Sent: 29 April 2013 22:51
> To: openstack-dev at lists.openstack.org
> Subject: Re: [openstack-dev] Continuous deployment - significant process
> change
> On 04/29/2013 05:04 PM, Robert Collins wrote:
> > We had a process track session about bringing in upstream continuous
> > deployment for openstack.
> > https://etherpad.openstack.org/HavanaContinuousDeployment
> >
> > I suspect that while the session was good with both deployer and
> > distributor attendees, we need to do more to make it happen, as it
> > impinges on review / testing / backwards compat requirements for every
> > project.
> >
> > Note that CD doesn't require no-downtime deployments, CD is about
> > being able to adopt *any arbitrary revision of trunk* at *any point in
> > time*. The engineering required to do deployments without disruption
> > is beneficial to both CD and per-release deployments.
> >
> > Here are the key takeaways we came up with:
> >  * No more big landings [except the purely mechanical]. Set a hard
> > limit - maybe 500 lines of diff. Big landings are more risky per line
> > of diff than small ones due to reviewer cognitive overhead - reviewers
> > get non-linearly less effective the larger the review.
> >
> I don't think we can set a # of lines that always makes sense.  However,
> I feel like in Nova we already do a nice job of pushing back hard on
> large patches in favor of breaking them up into a reasonable patch
> series.
> https://wiki.openstack.org/wiki/GitCommitMessages
> So, at least for Nova, this is business as usual.
> >  * CD can be done many ways; we need to gate the *specific* ways that
> > upstream adopts, as soon as possible. Thats a -infra thing, and there
> > are already discussions on it. We don't need to support *every
> > possible config for CD*. Organisations interested in a particular
> > configuration(s) will need to contribute resources to permit
> > gate-quality checks of those configurations.
> >
> >  * No more cramming: when a freeze is happening, anything that is
> > 'land this for the release' has to be pushed back on -really hard-. If
> > its not ready, it's not ready.
> >
> >  * -never- choose to break something that is neither experimental nor
> > deprecated since the last release. If an accident happens, correct it
> > as quickly as possible.
> >
> >  * Land features disabled by default. Such disabled features are
> > experimental and we don't need to be so careful - we can in fact
> > remove them entirely if they turn out to be bad idea - when they
> > become supported (individual teams can define this we think) they
> > can't be broken again though: they are now part of the product.
> >
> >  * 'To break' means just that - it could be an exception, it could be
> > a massive jump in DB utilisation, or latency. Whatever our criteria
> > are for 'fit for use', breaking something stops it being fit for use
> > in *existing deployed environments*.
> Most of this seems good.  It seems as if there's tension between:
> 1) Ensuring utmost quality so you can deploy any revision
> and
> 2) Being less careful with some new features and justifying it by having
> them disabled by default.
> I don't see the value in this disabled-by-default thing.  Another one of
> your points stresses the fact that if something's not ready, it's not
> ready, which I definitely agree with.

I think this article gives a good argument for "disabled-by-default".  It is a valid way to retain small commits to the code base which are easily reviewable whilst enabling developers to continue development on the feature in a safe way.  Without this you may lose the ability to keep commits small enough to be digestible to reviewers.


> --
> Russell Bryant
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Jon-Paul Sullivan ☺ Cloud Services - @hpcloud

Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park, Galway.
Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John Rogerson's Quay, Dublin 2. 
Registered Number: 361933
The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender.

To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as "HP CONFIDENTIAL".

More information about the OpenStack-dev mailing list