[openstack-dev] Continuous deployment - significant process change

Joshua Harlow harlowja at yahoo-inc.com
Mon Apr 29 23:40:51 UTC 2013

I agree with russell.

Although I was the one in that session that didn't get the land code
disabled/half-way complete as a way to avoid using feature branches. I
always thought landing a feature that was disabled and half-complete is
pretty similar to a feature branch in concept (except with a feature
branch u can reject it as a whole and with a inline disabled/half-way
complete code in master u can't rip it out). But to each there own, I can
understand getting code in early and often (due to change amounts and
rebasing...) so maybe its a case-by-case basis.

I think that with a proper prototype (to show others) and then proper
planning around the patch-sets that you can get something in that is in a
workable state instead of a half/way state.


On 4/29/13 3:48 PM, "Russell Bryant" <rbryant at redhat.com> wrote:

>On 04/29/2013 06:39 PM, Sullivan, Jon Paul wrote:
>>> -----Original Message-----
>>> From: Russell Bryant [mailto:rbryant at redhat.com]
>>> Sent: 29 April 2013 22:51
>>> To: openstack-dev at lists.openstack.org
>>> Subject: Re: [openstack-dev] Continuous deployment - significant
>>> change
>>> On 04/29/2013 05:04 PM, Robert Collins wrote:
>>>> We had a process track session about bringing in upstream continuous
>>>> deployment for openstack.
>>>> https://etherpad.openstack.org/HavanaContinuousDeployment
>>>> I suspect that while the session was good with both deployer and
>>>> distributor attendees, we need to do more to make it happen, as it
>>>> impinges on review / testing / backwards compat requirements for every
>>>> project.
>>>> Note that CD doesn't require no-downtime deployments, CD is about
>>>> being able to adopt *any arbitrary revision of trunk* at *any point in
>>>> time*. The engineering required to do deployments without disruption
>>>> is beneficial to both CD and per-release deployments.
>>>> Here are the key takeaways we came up with:
>>>>  * No more big landings [except the purely mechanical]. Set a hard
>>>> limit - maybe 500 lines of diff. Big landings are more risky per line
>>>> of diff than small ones due to reviewer cognitive overhead - reviewers
>>>> get non-linearly less effective the larger the review.
>>> I don't think we can set a # of lines that always makes sense.
>>> I feel like in Nova we already do a nice job of pushing back hard on
>>> large patches in favor of breaking them up into a reasonable patch
>>> series.
>>> https://wiki.openstack.org/wiki/GitCommitMessages
>>> So, at least for Nova, this is business as usual.
>>>>  * CD can be done many ways; we need to gate the *specific* ways that
>>>> upstream adopts, as soon as possible. Thats a -infra thing, and there
>>>> are already discussions on it. We don't need to support *every
>>>> possible config for CD*. Organisations interested in a particular
>>>> configuration(s) will need to contribute resources to permit
>>>> gate-quality checks of those configurations.
>>>>  * No more cramming: when a freeze is happening, anything that is
>>>> 'land this for the release' has to be pushed back on -really hard-. If
>>>> its not ready, it's not ready.
>>>>  * -never- choose to break something that is neither experimental nor
>>>> deprecated since the last release. If an accident happens, correct it
>>>> as quickly as possible.
>>>>  * Land features disabled by default. Such disabled features are
>>>> experimental and we don't need to be so careful - we can in fact
>>>> remove them entirely if they turn out to be bad idea - when they
>>>> become supported (individual teams can define this we think) they
>>>> can't be broken again though: they are now part of the product.
>>>>  * 'To break' means just that - it could be an exception, it could be
>>>> a massive jump in DB utilisation, or latency. Whatever our criteria
>>>> are for 'fit for use', breaking something stops it being fit for use
>>>> in *existing deployed environments*.
>>> Most of this seems good.  It seems as if there's tension between:
>>> 1) Ensuring utmost quality so you can deploy any revision
>>> and
>>> 2) Being less careful with some new features and justifying it by
>>> them disabled by default.
>>> I don't see the value in this disabled-by-default thing.  Another one
>>> your points stresses the fact that if something's not ready, it's not
>>> ready, which I definitely agree with.
>> I think this article gives a good argument for "disabled-by-default".
>>It is a valid way to retain small commits to the code base which are
>>easily reviewable whilst enabling developers to continue development on
>>the feature in a safe way.  Without this you may lose the ability to
>>keep commits small enough to be digestible to reviewers.
>> http://rc3.org/2013/03/03/dont-get-stuck/
>I think you can develop a feature in its entirety with a series of
>reasonably sized changes with a well designed patch set.
>Once code gets in to the tree, it becomes a burden on the project to
>maintain that code, even if disabled by default.  If your feature isn't
>done, I don't want it in the tree.
>Russell Bryant
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org

More information about the OpenStack-dev mailing list