[openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers

Sylvain Bauza sbauza at redhat.com
Mon Sep 8 16:23:40 UTC 2014


Le 08/09/2014 18:06, Steven Dake a écrit :
> On 09/05/2014 06:10 AM, Sylvain Bauza wrote:
>>
>> Le 05/09/2014 12:48, Sean Dague a écrit :
>>> On 09/05/2014 03:02 AM, Sylvain Bauza wrote:
>>>> Le 05/09/2014 01:22, Michael Still a écrit :
>>>>> On Thu, Sep 4, 2014 at 5:24 AM, Daniel P. Berrange
>>>>> <berrange at redhat.com> wrote:
>>>>>
>>>>> [Heavy snipping because of length]
>>>>>
>>>>>> The radical (?) solution to the nova core team bottleneck is thus to
>>>>>> follow this lead and split the nova virt drivers out into separate
>>>>>> projects and delegate their maintainence to new dedicated teams.
>>>>>>
>>>>>>    - Nova becomes the home for the public APIs, RPC system, database
>>>>>>      persistent and the glue that ties all this together with the
>>>>>>      virt driver API.
>>>>>>
>>>>>>    - Each virt driver project gets its own core team and is 
>>>>>> responsible
>>>>>>      for dealing with review, merge & release of their codebase.
>>>>> I think this is the crux of the matter. We're not doing a great 
>>>>> job of
>>>>> landing code at the moment, because we can't keep up with the review
>>>>> workload.
>>>>>
>>>>> So far we've had two proposals mooted:
>>>>>
>>>>>    - slots / runways, where we try to rate limit the number of things
>>>>> we're trying to review at once to maintain focus
>>>>>    - splitting all the virt drivers out of the nova tree
>>>> Ahem, IIRC, there is a third proposal for Kilo :
>>>>   - create subteam's half-cores responsible for reviewing patch's
>>>> iterations and send to cores approvals requests once they consider the
>>>> patch enough stable for it.
>>>>
>>>> As I explained, it would allow to free up reviewing time for cores
>>>> without loosing the control over what is being merged.
>>> I don't really understand how the half core idea works outside of a 
>>> math
>>> equation, because the point is in core is to have trust over the
>>> judgement of your fellow core members so that they can land code when
>>> you aren't looking. I'm not sure how I manage to build up half trust in
>>> someone any quicker.
>>
>> Well, this thread is becoming huge so that's becoming hard to follow 
>> all the discussion but I explained the idea elsewhere. Let me just 
>> provide it here too :
>> The idea is *not* to land patches by the halfcores. Core team will 
>> still be fully responsible for approving patches. The main problem in 
>> Nova is that cores are spending lots of time because they review each 
>> iteration of a patch, and also have to look at if a patch is good or 
>> not.
>>
>> That's really time consuming, and for most of the time, quite 
>> frustrating as it requires to follow the patch's life, so there are 
>> high risks that your core attention is becoming distracted over the 
>> life of the patch.
>>
>> Here, the idea is to reduce dramatically this time by having teams 
>> dedicated to specific areas (as it's already done anyway for the 
>> various majority of reviewers) who could on their own take time for 
>> reviewing all the iterations. Of course, that doesn't mean cores 
>> would loose the possibility to specifically follow a patch and bypass 
>> the halfcores, that's just for helping them if they're overwhelmed.
>>
>> About the question of trusting cores or halfcores, I can just say 
>> that Nova team is anyway needing to grow up or divide it so the 
>> trusting delegation has to be real anyway.
>>
>> This whole process is IMHO very encouraging for newcomers because 
>> that creates dedicated teams that could help them to improve their 
>> changes, and not waiting 2 months for getting a -1 and a frank reply.
>>
>>
> Interesting idea, but having been core on Heat for ~2 years, it is 
> critical to be involved in the review from the beginning of the patch 
> set.  Typically you won't see core reviewer's participate in a review 
> that is already being handled by two core reviewers.
>
> The reason it is important from the beginning of the change request is 
> that the project core can store the iterations and purpose of the 
> change in their heads.  Delegating all that up front work to a 
> non-core just seems counter to the entire process of code reviews. 
> Better would be reduce the # of reviews in the queue (what is proposed 
> by this change) or trust new reviewers "faster".  I'm not sure how you 
> do that - but this second model is what your proposing.
>
> I think one thing that would be helpful is to point out somehow in the 
> workflow that two core reviewers are involved in the review so core 
> reviewers don't have to sift through 10 pages of reviews to find new 
> work.
>

Now that the specs repo is in place and has been proved with Juno, most 
of the design stage is approved before the implementation is going. If 
the cores are getting more time because they wouldn't be focused on each 
single patchset, they could really find some patches they would like to 
look at, or they could just wait for the half-approvals from the halfcores.

If a core thinks that a patch is enough tricky for looking at each 
iteration, I don't see any bad things. At least, it's up to the core 
reviewer to choose which patches he could look at, and he would be more 
free than if the slots proposal would be there.

I'm a core from a tiny project but I know how time consuming it is. I 
would really enjoy if I could delegate some of my review time to people 
willing to. That's also a good opportunity for seeing whose people would 
be good for cores, because they would be more visible as halfcores, 
provided they would not only review in mass, but also participate to the 
design discussions etc.


We all know that the review metrics mean nothing, so we can't trust 
people even if they're doing lots of reviews. By creating a middle 
position for reviewing, that would focus the attention on their work, 
without aleviating what would be *really* approved. Don't leave me speak 
about Ben Parker from Spiderman, and how power and responsibilities are 
tied ;-)

-Sylvain


> Regards,
> -steve
>
>> As I said elsewhere, I dislike the slots proposal because it sends to 
>> the developers the message that the price to pay for contributing to 
>> Nova is increasing. Again, that's not because you're prioritizing 
>> that you increase your velocity, that's 2 distinct subjects.
>>
>> -Sylvain
>>
>>
>>>     -Sean
>>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




More information about the OpenStack-dev mailing list