Open Stack

On 08/12/2014 05:54 AM, Daniel P. Berrange wrote:
>> I am less concerned about the contents of this patch, and more concerned
>> with how such a big de facto change in nova policy (we accept untested code
>> sometimes) without any discussion or consensus. In your comment on the
>> revert [2], you say the 'whether not-CI-tested features should be allowed
>> to be merged' debate is 'clearly unresolved.' How did you get to that
>> conclusion? This was never brought up in the mid-cycles as a unresolved
>> topic to be discussed. In our specs template we say "Is this untestable in
>> gate given current limitations (specific hardware / software configurations
>> available)? If so, are there mitigation plans (3rd party testing, gate
>> enhancements, etc)" [3].  We have been blocking untested features for some
>> time now.
> 
> That last lines are nonsense. We have never unconditionally blocked untested
> features nor do I recommend that we do so. The specs template testing allows
> the contributor to *justify* why they think the feature is worth accepting
> despite lack of testing. The reviewers make a judgement call on whether the
> justification is valid or not. This is a pragmmatic approach to the problem.

That has been my interpretation and approach as well: we strongly prefer
functional testing for everything, but take a pragmatic approach and
evaluate proposals on a case by case basis.  It's clear we need to be a
bit more explicit here.

-- 
Russell Bryant