Open Stack

On 15 November 2013 03:27, Sean Dague <sean at dague.net> wrote:

> here, but in general). Having a lot of eager contributors, a small
> number of people that understand the implications to Nova core, and a
> finite time to build shared culture to make sure it's not a bag of parts
> that doesn't hold together....
>
> **it's just an algebra problem to solve.**
>
> I get that people don't like that it takes a long time to get code into
> Nova. But with so many people doing CD now, it really feels like the
> cost of a false positive (landing code that never should) is way higher
> than the cost of a false negative (keeping code out when it could have
> come in).

I see it the other way: There are two factors.

 - the CI and CD work we're doing is our safety net. Thats what stops
coarse issues from landing at all.
 - subtle issues will always land - it's a fact of life - CD deployers
want a very low TTD and TTR (time to detect, time to resolve) pair:
the faster we detect an issue *and* resolve it, the better. Review
latency is clearly a big part of the timing for the resolve side of
the process.

> I get solving algebra problems is easy, solve for X, look here is
> perfect solution, add 17.3 new nova cores and everything is fixed. But
> that's not the real world. Relationships matter. If they didn't we
> wouldn't *vote* on new core members, we'd just auto promote people.

Math won't fix it, but understanding the problem clearly is useful I
think: this *is* a scaling problem, because folk are burning out. They
are leaving the nova review team, coming back and dropping again. (Ok,
sample size of one comes to my head :)). "We need more reviewers" is a
motherhood statement like 'apple pie is nice' : but a clear - *this*
is why we are drowning, and *these* are the requirements - the human
requirements - we look for in core reviewers : that helps understand
the problem. If the problem really is that the ratio of 'ready to be
good reviewers' and 'not ready but can cut code' is too low - then we
need to work on training folk up to be better reviewers. Thats a long
term thing, and to invest in it.. we need shared agreement that its
the problem, that it's solvable, and how to solve it. Anything that
isn't 'solve today' needs this sort of shared understanding and
committment.

That said you are absolutely right that numbers can't determine 'X is
a good reviewer', but they can show 'X is unlikely to be a good
reviewer' - but the scary thing is that humans are actually *very bad*
at tracking long term information and making considered decisions. If
you want a terrifying view on this read 'thinking, fast and slow' some
day. http://www.amazon.com/gp/product/B005MJFA2W/ref=kinw_myk_ro_title


> In my role in QA I've experienced the different cultures across
> projects, the level of coherency in their core teams, and the quality of
> software that comes out the other end. Honestly, Nova is doing a better
> job than just about anyone else, when we quantify it on the metric I
> think we all most care about: final quality. So people that aren't
> steeped in that culture, coming up with ways to *fix* Nova, strikes me
> as something not only a little misguided, but potentially really
> dangerous second order consequences.

I don't want to fix Nova - I'm following up on discussions that where
held at the Summit - with Russell - and seeing where it leads. I do
want to help reduce the wear and tear on core reviewers, and to
improve the efficiency with which we develop OpenStack, where that is
compatible with good architecture, design, and software quality.

> If you spend a lot of time in the Nova track at summit you see what that
> shared culture creates. There is a lot of magic smoke in that box. Could
> there be improvements, absolutely, and the most inner folks in Nova talk
> about this all the time, trying to come up with new models. But I really
> don't think this is a fix-it-with-math problem.

I think I agree; I hope you'll tolerate a little more modelling as we
get to grips with the symptoms and shape of the problem though.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud