Open Stack

> We do already give you this option via the removal of tests that are
> invalid/flaky/not useful. I do worry that if we give a complete end
> around the CI system it will be quickly abused.

Absolutely agree, humans are not good at making these decisions. Despite
"trust" in the core team, and even using a less-loaded word than
"abuse," I really don't think that even allowing the option to override
flaky tests by force merge is the right solution (at all).

> I tried to make this point in my previous email, but I think we are
> still fumbling around it. If we provide mechanisms to end around flaky
> CI instead of fixing flaky CI the end result will be flakier CI. I'm
> not convinced that we'll be happier with any mechanism that doesn't
> remove the -1 from happening in the first place. Instead the problems
> will accelerate and eventually we'll be unable to rely on CI for
> anything useful.

Agreed. Either the tests are useful or they aren't. Even if they're not
very reliable, they might be useful in causing pain because they
continue to highlight flaky behavior until it gets fixed.

--Dan