Open Stack

On Fri, Oct 3, 2014 at 9:05 AM, Jay Pipes <jaypipes at gmail.com> wrote:
> On 10/03/2014 10:06 AM, Chris Dent wrote:
>>
>> On Fri, 3 Oct 2014, Anne Gentle wrote:
>>
>>> I'm reading and reading and reading and my thoughts keep returning to,
>>> "we're optimizing only for dev." :)
>>
>>
>> Yes, +many.
>
>
> plus infinity.
>
>> In my reading it seems like we are trying to optimize the process for
>> developers which is exactly the opposite of what we want to be doing if
>> we want to address the perceived quality problems that we see. We should
>> be optimizing for the various user groups (which, I admit, have been
>> identified pretty well in some of the blog posts).
>
>
> Precisely.
>
>> This would, of course, mean enhancing the docs (and other cross
>> project) process...
>
>
> Yep! This is something I harped on quite a bit in my second blog post.
>
>> At the moment we're trying to create governance structures that
>> incrementally improve the existing model for how development is
>> being done.
>>
>> I think we should consider more radical changes, changes which allow
>> us to work on what users want: an OpenStack that works.
>
>
> Yay!

This ++

Also, I think what I've proposed does this. Because, as Sean already
pointed out, our user surveys still say that most users are trying to
get compute+identity+images+network when they try to get "an OpenStack
that works". Tightening the focus of the horizontally-scaling teams
(qa, docs) to focus on that for a bit will help deliver a better
experience to users.

Expecting those horizontal teams to scale to accomodate every project
under the big-tent umbrella is ludicrous, and I don't think anyone's
expecting that to actually work out. I would much rather make it very
clear to projects which are outside of what I keep wanting to call
"ring compute" that they need to do their own docs, qa, etc - but they
should be following the same workflow patterns and participating with
the qa and doc teams working on "ring compute". That collaboration
makes them part of OpenStack, regardless of whether they co-gate with
Nova or release on the same cadence as Nova.


>
>> To do that I think we need to figure out two things:
>>
>> * how to fail faster
>> * how to stop thinking of ourselves as being on particular projects
>
> More yay!



>
>> I got hired to work on telemetry, but I've managed to do most of my
>> work in QA related things because what's the point of making new
>> stuff if you can't test it reliably? What I'd really like to say my
>> job is is "making OpenStack the best it possibly can be".
>>
>> If we keep focusing on the various services as entangled but separate
>> and competing interests rather than on how to make OpenStack good, we're
>> missing the point and the boat.

That's not the point in my head, so let me clarify:

I'm essentially saying this:

Let's STOP co-gating the world, and only co-gate on non-optional
functional dependency boundaries between projects, when those projects
choose to co-gate with each other because the developers trust each
other.

By running a project's functional tests only against changes in that
project, and encouraging (but leaving it up to the projects to decide
whether they mutually run) two-service integration tests where there
is an actual functional dependency between two projects, means each
project actually has to unwrap its assumptions about the other
project's behavior. We'll quickly learn where there are unstated
assumptions about behavior as projects break other projects which they
aren't co-gating with, and from that, we'll start shoring up those
pain points for our users.

>>
>> Our job as developers is to make things easier (or at least
>> possible) for the people who use the stuff we build. Naturally we
>> want to make that as frictionless as possible, but not at the cost
>> of "the people's" ease.
>>
>> There are many perverse incentives in OpenStack's culture which
>> encourage people to hoard. For example it is useful to keep code in
>> one's own team's repository because the BPs, reviews and bugs which
>> reflect on that repository reflect on the value of the team.
>>
>> Who is that good for?
>>
>> So much of the talk is about trying to figure out how to make the
>> gate more resilient. No! How about we listen to what the gate is
>> telling us: Our code is full of race conditions, a memory pig,
>> poorly defined contracts, and just downright tediously slow and
>> heavy. And _fix that_.
>
>
> I think it's important to discuss both things: the gate structure (and
> various inefficiencies the gate *policies* create within the system) as well
> as the fragility and design of the code bases themselves.

++

>
>> What I think we need to do to improve is enhance the granularity at
>> which someone can participate. Smaller repos, smaller teams, cleaner
>> boundaries between things. Disconnected (and rolling) release cycles.
>> Achieve fail fast by testing in (much) smaller lumps before code
>> ever reaches the global CI. You know: make better tests locally that
>> confirm good boundaries. Don't run integration tests until the unit,
>> pep8 and in-tree functional tests have passed. If there is a
>> failure: exit! FAIL! Don't run all the rest of the tests uselessly.
>
>
> Yup, ++.
>

Huge ++ on this. And it's one of the motivations behind the gating
changes I'm proposing -- less dependency on undocumented behavior and
more stable APIs is better for our users.

>> We need to not conflate the code and its structure with the
>> structure of our governance. We need to put responsibility for the
>> quality of the code on to the people who make it, not big infra.
>> We need to make it easier for people to participate in that quality
>> making. And most importantly we need to make sure the users are
>> driving what we do and we need to make it far easier for them to do
>> that driving.
>>
>> Obviously there are many more issues than these, but I think some of
>> the above is being left out of the discussion, and this message
>> needs to stop.
>
>
> Amen, brother.
>
> -jay
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev