[openstack-dev] [Nova] What's holding nova development back?

Sean Dague sean at dague.net
Tue Sep 16 14:45:08 UTC 2014


On 09/16/2014 09:39 AM, Jay Pipes wrote:
> On 09/16/2014 04:12 AM, Daniel P. Berrange wrote:
>> On Tue, Sep 16, 2014 at 07:30:26AM +1000, Michael Still wrote:
>>> On Tue, Sep 16, 2014 at 12:30 AM, Russell Bryant <rbryant at redhat.com>
>>> wrote:
>>>> On 09/15/2014 05:42 AM, Daniel P. Berrange wrote:
>>>>> On Sun, Sep 14, 2014 at 07:07:13AM +1000, Michael Still wrote:
>>>>>> Just an observation from the last week or so...
>>>>>>
>>>>>> The biggest problem nova faces at the moment isn't code review
>>>>>> latency. Our
>>>>>> biggest problem is failing to fix our bugs so that the gate is
>>>>>> reliable.
>>>>>> The number of rechecks we've done in the last week to try and land
>>>>>> code is
>>>>>> truly startling.
>>>>>
>>>>> I consider both problems to be pretty much equally as important. I
>>>>> don't
>>>>> think solving review latency or test reliabilty in isolation is
>>>>> enough to
>>>>> save Nova. We need to tackle both problems as a priority. I tried
>>>>> to avoid
>>>>> getting into my concerns about testing in my mail on review team
>>>>> bottlenecks
>>>>> since I think we should address the problems independantly / in
>>>>> parallel.
>>>>
>>>> Agreed with this.  I don't think we can afford to ignore either one
>>>> of them.
>>>
>>> Yes, that was my point. I don't mind us debating how to rearrange
>>> hypervisor drivers. However, if we think that will solve all our
>>> problems we are confused.
>>>
>>> So, how do we get people to start taking bugs / gate failures more
>>> seriously?
>>
>> I think we should have formal "Bug squash wednesdays"  (or pick another
>> day). By this I mean that the core reviewers will focus their attention
>> on just reviews that are related to bug fixing. They will also try to
>> work on bugs if they have time and encourage everyone else involved in
>> Nova todo the same. We'd have a team of people in the Nova IRC channel
>> to publicise & co-ordinate bug squashing, perhaps  with a list of top
>> 20 bugs we want to attack this week. I wouldn't focus just on gate bugs
>> here since many a pretty darn hard & so would put off many people. Have
>> a mix of bugs of varying difficulties to point people to. Make this a
>> regular fortnightly or even weekly event which we publicise in advance
>> on mailing lists, etc.
> 
> +1, I've suggested similar in the past.

+1 a weekly event would be great.

I've spent the bulk of the last 2 weeks in the Nova bug tracker, and
it's pretty interesting what's in there. Lots of stuff we should be
fixing. Lots of really old gorp that we should shed because it's not
helping. Also lots of inconsistencies in how triage is happening because
it's not happening regularly enough.

Plus, now that we are at 0 bugs in the New state in Nova, it's actually
kind of sane to stay on top of that, and keep our New state empty. Not
that it fixes everything, but it does prevent a bunch of gorp getting
added to the pile as probably 1/2 - 1/3 of inbound bugs... aren't.

	-Sean

-- 
Sean Dague
http://dague.net



More information about the OpenStack-dev mailing list