[openstack-dev] Continuous deployment - significant process change

Clint Byrum clint at fewbar.com
Tue Apr 30 22:33:27 UTC 2013

On 2013-04-30 14:52, Russell Bryant wrote:
> On 04/30/2013 04:59 PM, Clint Byrum wrote:
>> On 2013-04-30 13:12, Sean Dague wrote:
>>> On 04/30/2013 03:55 PM, Clint Byrum wrote:
>>> <snip>
>>>> I understand that this is a straw-man. I know that other things can
>>>> happen. But I see a lot of development happen under the stable 
>>>> release
>>>> process scenario above. I fail to see where having new code arrive
>>>> incrementally in small batches and protected by feature flags is 
>>>> going
>>>> to make anything worse. Those things, however, should make 
>>>> continuous
>>>> delivery of OpenStack much smoother.
>>> Right, it's a straw man, which doesn't help generate any clarity. :)
>>> I think instead of discussing philosophy here we should talk about
>>> specifics that happened in the source tree that caused deployers 
>>> pain,
>>> and take lessons learned out of them.
>>> I think the reality is that openstack tree and process is very close
>>> to what the CD folks want. The only thing that's causing some 
>>> allergy
>>> among the core teams is the idea of features coming in in an off
>>> state, versus coming in default on so they get exercised by our 
>>> normal
>>> system.
>> Thats exactly my point. Large changes are not being exercised by any
>> large scale system now while they're baking in a branch, so making 
>> them
>> enabled by feature flags versus by commit to trunk does not regress
>> that, but it does allow deployers to test both code paths and deploy
>> both without having to maintain branches of the software.
> My biggest concern with this is applying it to an open source project
> with an unmanaged workforce.  Thierry captured it well when he said:
>   Personally my experience of OpenStack project management makes me 
> fear
>   that your proposal ignores the reality of our unmanaged development
>   workforce: there is no way we can be sure that a feature will 
> actually
>   be completed. So you might end up having disabled code in 
> releases...
>   or force the removal of the already-landed parts at the very last
>   moment, which is disruptive to release quality and also sounds more
>   disruptive to CD than what you are trying to protect against.
> I'm really worried about the transition of ownership going from the
> submitter to the project as a whole much earlier in the process.  I'd
> like to have a high confidence level that it's worth having this code
> for the long run before it starts coming in.  That to me requires it
> baking out of tree and getting completed before we accept it.

What is the negative impact of having disabled code in a release? I 
think we would need a good reason before we would back accepted code 
out. If the code is just sitting there, undocumented and turned off, not 
interfering with anything, I'm having a hard time thinking of a logical 
reason to back it out.

The fear that somebody will drop half-baked code into the project, and 
then never return, is valid. However the impact of such a heinous act is 
mitigated by the requirement for feature flags. If the half-baked idea 
is never completed, so be it, back it out as part of normal code 

Didn't this exact scenario happen with hyperV?

More information about the OpenStack-dev mailing list