Open Stack

On 09/10/2014 06:11 PM, Jay Pipes wrote:
> 
> 
> On 09/10/2014 05:55 PM, Chris Friesen wrote:
>> On 09/10/2014 02:44 AM, Daniel P. Berrange wrote:
>>> On Tue, Sep 09, 2014 at 05:14:43PM -0700, Stefano Maffulli wrote:
>>
>>>> I have the impression this idea has been circling around for a while
>>>> but
>>>> for some reason or another (like lack of capabilities in gerrit and
>>>> other reasons) we never tried to implement it. Maybe it's time to think
>>>> about an implementation. We have been thinking about mentors
>>>> https://wiki.openstack.org/wiki/Mentors, maybe that's a way to go?
>>>> Sub-team with +1.5 scoring capabilities?
>>>
>>> I think that setting up subteams is neccessary to stop us imploding but
>>> I don't think it is enough. As long as we have one repo we're forever
>>> going to have conflict & contention in deciding which features to
>>> accept,
>>> which is a big factor in problems today.
>>
>> If each hypervisor team mostly only modifies their own code, why would
>> there be conflict?
>>
>> As I see it, the only causes for conflict would be in the shared code,
>> and you'd still need to sort out the issues with the shared code even if
>> you split out the individual drivers into separate repos.
> 
> a) Sorting out the common code is already accounted for in Dan B's
> original proposal -- it's a prerequisite for the split.
> 
> b) The conflict Dan is speaking of is around the current situation where
> we have a limited core review team bandwidth and we have to pick and
> choose which virt driver-specific features we will review. This leads to
> bad feelings and conflict.
> 
> c) It's the impact to the CI and testing load that I see being the
> biggest benefit to the split-out driver repos. Patches proposed to the
> XenAPI driver shouldn't have the Hyper-V CI tests run against the patch.
> Likewise, running libvirt unit tests in the VMWare driver repo doesn't
> make a whole lot of sense, and all of these tests add a
> not-insignificant load to the overall upstream and external CI systems.
> The long wait time for tests to come back means contributors get
> frustrated, since many reviewers tend to wait until Jenkins returns some
> result before they review. All of this leads to increased conflict that
> would be somewhat ameliorated by having separate code repos for the virt
> drivers.

So I haven't done the math recently, what do you expect the time savings
to be here? Because unit tests aren't run by 3rd party today.

On my fancy desktop (test time including testr overhead):
 * tox -epy27: 330s
 * tox -epy27 libvirt: 18s
 * tox -epy27 vmware: 9s
 * tox -epy27 xen: 18s
 * tox -epy27 hyperv: 13s

The testr overhead is about 8s for discovery (yes, I do realize that's
probably more than it should be, that's a different story), so we'd be
looking at a reduction of about 10% of the total run time of unit tests
if we don't have the virt drivers in tree. That's not very much.

The only reason we're asking 3rd party CI folks to test everything... is
policy. I don't think it's a big deal to only require them to test
changes that hit their driver. Just decide that.

The conflict isn't going to go away, it's going to now exist on
integration, where there isn't a single core team to work through it in
a holistic way. This is hugely more painful place to pay it.

...

Right now the top of the gate is 26 hrs.

One of the reasons that that continues to grow and get worse over time
is related to the total # of git trees that we have to integrate that
don't have common core teams across them that understand their
interactions correctly. I firmly believe that anything that creates more
git trees that we have to integrate after the fact makes that worse. I
believe the 10+ oslo lib trees have made this worse. I believe
continuing to add new integrated projects has made this worse. And I
believe that a virt driver split by any project will make it worse, if
we expect to test that code upstream.

The docker driver in stackforge has been a success in merging docker
code. It's been much less of a success in terms of making it easy for
anyone to run it, use it, or for us to get on common ground for a
containers service moving forward.

The bulk of the folks that would be on the driver teams don't really
look at failures that expose in the gate. So I have a lot of trepidation
about the claims that this will make integration better by folks that
don't spend a lot of time looking and helping on our current integration.

...

That being said, I'm entirely pro cleaning up the virt interfaces as a
matter of paying down debt. That was a blueprint when I first joined the
project, that died on the vine somewhere. I think more common
infrastructure for virt drivers would be a good thing, and make the code
a lot more understandable. And as it's the prereq for any of this, so
let's do it.

Why don't we start with "let's clean up the virt interface and make it
more sane", as I don't think there is any disagreement there. If it's
going to take a cycle, it's going to take a cycle anyway (it will
probably take 2 cycles, realistically, we always underestimate these
things, remember when no-db-compute was going to be 1 cycle?). I don't
see the need to actually decide here and now that the split is clearly
at least 7 - 12 months away. A lot happens in the intervening time.

	-Sean

-- 
Sean Dague
http://dague.net