Open Stack

On 09/03/2014 11:37 AM, Joe Gordon wrote:
> As you all know, there has recently been several very active discussions
> around how to improve assorted aspects of our development process. One idea
> that was brought up is to come up with a list of cycle goals/project
> priorities for Kilo [0].
>
> To that end, I would like to propose an exercise as discussed in the TC
> meeting yesterday [1]:
> Have anyone interested (especially TC members) come up with a list of
> what they think the project wide Kilo cycle goals should be and post
> them on this thread by end of day Wednesday, September 10th. After which
> time we can begin discussing the results.
> The goal of this exercise is to help us see if our individual world
> views align with the greater community, and to get the ball rolling on a
> larger discussion of where as a project we should be focusing more time.

Again, thanks for bringing up this important topic. Here's my take on 
the cross-project, larger problem domains that I believe we should focus 
as a community on in Kilo:

1) Apply Drano to the current blockage in our governance policies and 
program pipeline

I'm finalizing a blog post about the community and governance policy 
issues that I see, but the Cliff's Notes version is that I think we should:

  a) get rid of the official incubated status
  b) adopt strictly the OpenStack Layers taxonomy for classification of 
projects (and get rid of Programs entirely)
  c) loosen the restrictions on projects living in the openstack/ code 
namespace
  d) replace the official integrated project status with finer-grained 
information that is actually useful to deployers

2) Apply LiquidPlumber to the current testing infrastructure gate

I think it's clear that there is significant pain currently felt by 
contributors across the board, and a large amount of pain experience by 
folks setting up and maintaining external CI systems. We need to focus 
on how to make our gating systems as efficient as possible, with as few 
false failures and as few non-relevant test runs as possible.

I believe Dan Berrange's proposal to split out the virt drivers in Nova 
(all of them) into separate repositories will be a huge win here in 
terms of reducing false positives and avoiding running external CI 
systems for drivers that are not related to a patch in another driver. I 
think Neutron and Cinder would be able to alleviate some gate pain by 
doing a similar effort.

I think pulling functional tests into the individual projects will also 
help to give the gate a bit of unclogging, since functional tests can be 
removed from the lengthier devstack-gate-based Tempest integration tests.

We need to continue making progress in documenting our upstream CI 
systems, how the gate works, how to diagnose and debug gate issues, how 
to bootstrap a developer environment, and how to set up external CI systems.

Finally, I believe there are significant things that can be done to 
reduce both unit and functional test run time. Personally, I'm almost 
finished with a branch in Nova that replaces the functional tests in 
/nova/tests/integrated/api_samples/ with a separate runner that only 
does environment setup once instead of on each test method setUp(). The 
runtime for testing api_samples goes from >5 minutes to <30 seconds on 
my machines. Similar things can and should be looked at for other 
projects to give our gate something of an enema.

3) Apply Mr. Clean to the crufty interfaces that litter our projects

Folks have discussed this at length, and I'm also finishing up another 
longish etherpad and ML discussion about refactoring in Nova's 
api->conductor->scheduler internal interfaces, but the bottom line is 
that there are a significant number of internal interfaces that need to 
be cleaned up, versioned, and "objectified" before splitting out any of 
the drivers or subsystems can be done easily.

We need to pay down the technical debt that has built up, and 
refactoring crufty interfaces is one good way to do that.

Thanks,
-jay