Open Stack

Hey Nikhil, found this all very interesting. Given your next to last
paragraph where you ask for a "judiciary" it sounds like there is
more going on here than I'm aware of, so apologies if my comments
are not entirely germane: I found some of your analysis very
interesting and related to things I've been thinking about since
joining the community two years ago, and wanted to respond. I've
probably missed some nuance in my effort to pick out the critical bits.

The analysis you do to lay in the background (OpenStack process is slow,
code is communication, people make a lot of assumptions that make it
harder to converge on action) is rather astute but from my standpoint
the apparent conclusion you make (to make up for inefficiencies in
existing communication styles let's have more synchronous communication
and synchronous participation) is unfortunate for a project at the scale
(on many dimensions) of OpenStack.

The fundamental problem is that we don't have shared understanding
and we don't do what is needed to build share understanding. In fact
we actively work against building it. TL;DR: When we are apart we need
to write more; when we are together we need to talk more but with less
purpose.

I mostly agree (and have been crowing) that:

> many strong influencers
> are missing on the most important need for OpenStack -- efficient way of
> communication.

but disagree that getting people into so-called "high bandwidth" meetings
is the solution. Neither audio/video meetings nor IRC meetings are
of any use unless the participants have already established the basic
shared understanding that is required for the communication that
happens in the meeting to make any sense.

Most people in the OpenStack don't have that shared understanding
and without that shared understanding the assumption problem you
describe compounds. People make a list of goals that they believe
are mutually shared but they are actually just a list of words for
which everyone has a different interpretation.

(Part of the reason, I think, that people have a tendency to be
disruptive in IRC meetings is because it is incredibly obvious that
no one is talking about the same thing even though they claim to be.
The meeting is already just so much noise, why not add more? People
act out because they are squirming in the face of lack of coherent
meaning. TC meetings are classic and sometime hilarious examples of
this: People talking over one another, ostensibly about the same topic,
but from entirely different foundations. Without the foundations there's
no real point having the conversations, especially in a medium where
the foundations can't be built.)

We could improve this situation by altering how we are using our
communication tools to be more oriented towards establishing the
foundations. In a past life we used to talk about this collaborative
process as a sequence of "shared":

* Developing shared language and share understanding is a prerequisite
   for
* Developing shared goals which are a prerequisite for
* Share actions

Shared language is developed through hanging out and shooting the
breeze, talking things over in the broad sense, finding common
ground. It is the phase where you figure out what matters in the
very broad sense. This is the sort of stuff that happens in the
hallways at summit, late at night in open ended conversations at
IRC. The undirected and unplanned sense-making.

This stage is difficult for OpenStack newcomers or short-timers but
because there are so many newcomers and short-timers it is critical
that we figure out ways to make it easier. There must be
discoverable and digestible (thus not etherpads nor IRC logs) artifacts
from those conversations, ways in which the oral history becomes the
written history. Yes, this means more work for the old-timers, but as
you correctly point out Nikhil: communicating this sort of stuff should
really be their primary activity.

This myth-building happens to some extent already but there is not
a strong culture in OpenStack of pointing people at the history.
It's more common as a newbie to rock up in IRC with a question and
get a brief synchronous answer when it would be better to be pointed
at a written artifact that provides contextualizing background.

It's important to note: At this stage being pointed at etherpads,
gerrit reviews and IRC logs won't cut it. We're talking here about
building shared language. There are two issues with those media:

* They are already embedded in context of the language (that is,
   they only make sense if you already "get it).
* The UX on etherpads, gerrit and IRC logs for extracting meaning is
   dismal. If you're sending people to raw etherpads to help them learn
   something you are a cruel person. Etherpads are references, not
   digests.

The current expense of summits is such that participants go in with an
eye to coming out of it with goals not realizing that they first
need to establish the shared language. As a commercially driven open
source project it's high time we as participants start using the
summit (and summit-like events) and the commercial resources correctly.
The reason for being there is to build shared language and to do that a
few things need to happen:

* Anyone who is expected to be making active contributions to
   OpenStack should be there.
* There should be far more loosely structured time where people talk
   about "what's this all about?" (building the big picture) instead
   of "how are we going to fix the frobnitz in the widget?" (working
   out the details of a single spec).

That probably sounds really expensive with very ambiguous return on
investment but hey, we're seeing predications of a multi-billion
dollar market coming soon. It takes money to make money. Sending
developers to summits and mid-cycles ought to be a cost doing
business.

Many of the artifacts from summit (and things like it) should be broad
strokes of "here's how things mean, and here's what matters".

We've become trapped into the idea of using f2f or synchronous time
for "working out the details of a single spec" because we haven't
got the shared language that makes talking about details effective.
If we get the shared language the details become so much easier to
deal with, in any medium.

(Will get back to the stages of "shared" in a moment but this seems
like an opportune moment to jump back into your content:)

> I think people prefer to use ML a lot and I am not a great fan of the
> same. It is a multi-cast way of communication and it has assumptions
> around time, space, intent of the audience & intent to actually read
> them. Same is for gerrit/etherpad.

I'm one of those people who prefers ML. The assumptions is makes
about time, space and intent are pretty clear:

* anyone can read it at any time
* it archives effectively, in context
* it can be referenced later
* given sufficient care it is thoughtful and reflective rather than
   reactive

This makes it the absolute clear winner over the synchronous
choices.

> Multi-cast medium of communication is more disruptive as it involves a
> possibility of divergence from the topic, strongly polarizing opinions
> due to the small possibility of catching of the intent. So, let us use
> it 'judiciously' and preferably only as a newspaper.

The reason this happens is because of the lack of shared
understanding that I've described above. People are expressing their
lack of understanding and their need to gain it by meandering all
over the map trying to find what matters. If this keeps happening in
whatever medium that is being used, it is a crystal clear sign that
people haven't got shared understanding so there is very little
point talking about shared goals yet: You might think you've got
goals, but they certainly aren't shared and it's highly likely they
are not understood.

My experience of most OpenStack communications is that the degree of
shared understanding on some very fundamental things (e.g. "What are
we building?") is extremely low. It's really not that surprising
that there is discomfort.

Once you do have some shared understanding then a variety of media
can be useful in establishing a shared goal. Again, I tend to prefer
asynchronous modes for this for the same reasons as before:

* It allows people to participate without being beholden to time.
* It keeps a linkable artifact of the discussion process.

Yes, of course, the process can be long and involve a fair amount
decoherence but that's the process: it is reaching towards a solid
crystallization of the ideas into agreed goals.

Voice-based meetings exclude people and usually allow the
traditional power holders to maintain their power. The odds of a
divergent (but good) idea entering are low.

IRC meetings are just chaos.

(IRC is good for two things:

* Shooting the breeze and turning strangers into colleagues.
* Dealing with emergent situations ("the gate's on fire, HALP!").

Using it as a question and answer domain allows and encourages knowledge
that ought to be persistent to stay ephemeral. Using it for meetings
of more than a very small number of co-languaging people (at least
without a talking stick) is just...I don't even...who ever thought that
was a good idea?)

> * I find it is very important for people who actually have a strong say
> in things to focus less on code or even individual reviews and focus
> more on awareness, collaboration and establishing convergence factor.
> Leave the nitty gritty details to those who have more bandwidth in hand
> for it.

I agree with this but to me it means that those people should be
writing a lot more summarizing email (so as to help other people
reach awareness).

> * I think if someone has a very strong say then they need to keep a
> (near to) synchronous communication to the development process. We need
> to keep the context, keep our convergence intact and move forward with a
> common understanding. Otherwise it is VERY disruptive for someone
> investing their time, money, energy and interest in OpenStack.

I disagree with this. Yes, we greatly need to reduce the latency
in feedback loops. However if we want all contributors to
participate in the learning and project improvement that is gained
from that feedback it _has_ to be written so that it can be consumed
asynchronously.

It seems like you are trying to privilege short term convergence at
the cost of long term convergence. That sounds like the action of a
desperate project that has gone so far down the path of not having
shared understanding that it is sacrificing the future. Please don't
do that. Stop. Breathe. Think about how to enhance informed
participation. You'll get more throughput if people understand and
agree why they are doing what they are doing.

So, now, back to goals.

If your participants have somehow managed to achieve some shared
understanding they can use that understanding when creating and
discussing goals. When they get together in IRC for a weekly
catchup and it is time to discuss "how's it going with the frobnitz"
rather then everyone squirming because everyone has a different idea
of what the frobnitz is (let alone how to fix it) people, although
they may not agree on the solution, will at least agree on the
problem domain so the scope of the discussion and the degree of what
you're calling "disruption" (and I would call "discomfort") goes down,
making room, finally, for shared action.

I'm going to leave it there. Not because I'm done but because
history makes it pretty clear that people in this community get
fussy near long emails. If you've made it this far, congratulations.

The summary: spend more time and energy establishing shared
understanding and documenting shared language. Do it in a way that
is inclusive of newcomers. Do it because it is the most important
part of the work you're doing, it is not overhead.

> Thank you for reading,

Thank you for _writing_. It is so much better to be able to consume
these issues in my own time than trying, for example, to try (and
fail) to extract something like this from observing an IRC meeting.

-- 
Chris Dent               (╯°□°)╯︵┻━┻            http://anticdent.org/
freenode: cdent                                         tw: @anticdent