Open Stack

On 01/13/2016 03:41 PM, Armando M. wrote:
> 
> 
> On 13 January 2016 at 11:24, Carl Baldwin <carl at ecbaldwin.net
> <mailto:carl at ecbaldwin.net>> wrote:
> 
>     Hi,
> 
>     I was looking at the most recent gate breakage in Neutron [1], fixed
>     by [2].  This gate breakage was held off for some time by the
>     upper-constraints.txt file.   This is great progress and I applaud it.
>     I'll continue to cheer on this effort.
> 
>     Now to the next problem.   If my assessment of this gate failure is
>     correct, the update to the upper-constraints file [3] was merged
>     without running all of the tests across all of the projects that would
>     be broken by bringing in this new constraint.  So, we still get
>     breakage and it is still (IMO) too often.
> 
> 
> This is my understanding too, but I let the infra and requirements gurus
> confirm.
>  
> 
>     As I see it, there are a couple of options.
> 
>     1) We run all tests under the upper-constraints control on all updates
>     to the upper constraints file like [2].  This would probably mean each
>     update has a very long list of tests and we would require that they
>     all be fixed before the upper constraint update can be merged.  This
>     seems like a difficult thing to coordinate all at once.
>     2) We handle upper-constraints much like we do the global requirements
>     updates.  We have the master and a bot that proposes updates to it out
>     to the individual projects.  This would create a situation where
>     projects are out of sync with the master but I think if we froze the
>     master early enough, we could have time to reconcile before release.
>     3) We continue to allow changes in the upper constraints to break
>     individual projects.
> 
> 
>     Are there options that I missed?  What is your opinion?  In my
>     opinion, gate breakage happens a bit too often and the effect on the
>     community is widespread.  I'd like to contain it even a little bit
>     more.
> 
> 
> I suppose another (not ideal) solution might be to use Depends-on
> cautiously. We could have filed a sentinel patch against the neutron
> repo in conjunction with the upper-constraints change. 
> 
> That said, I'd love to be a little more bullet proof.

The important thing to remember is that unless all test suites are 100%
passing, adding more test suites to jobs actually just makes the false
positive on fails go up. At which point it's now making it harder for
people to land good code. It also consumes overall test nodes, which
slows down development for everyone.

There is no full machine solution here. There are sets of helpers, and
then there is the need for reviewers to be cognizant of the downstream
effects of software releases, and that people writing tests be defensive
and only rely on actual contract behavior. Robustness on each side of
the fence.

	-Sean

-- 
Sean Dague
http://dague.net