[Openstack-operators] Atlanta Summit - More Ops? ;)

Tim Bell Tim.Bell at cern.ch
Mon Mar 31 18:04:42 UTC 2014

Thanks for the great input.

I'm trying to track the points raised in the etherpad https://etherpad.openstack.org/p/ATL-ops-unconference-RFC for the governance discussion at the summit.

The OSSG model would be an interesting one. While I agree with Jay's admin/operator distinction, I think there is nothing that stops us starting out with 'operator' including both roles and then splitting further later. I am keen on something light and flexible since the effort should be focussed on getting OpenStack better.

I do feel there is a major role distinction between operator and cloud consumer (i.e. someone who uses the API/CLIs to deliver function). There was an item on the OpenStack board meeting last time to discuss how to serve the consumer role better since we are starting to establish an operator community but there is a need to start on the next wave.

The good news is that the by-laws for the user committee are very open. We can establish the structure we wish subject to agreement of JC, Ryan and myself. Thus, there is a real opportunity to get things going in Atlanta if we can find the right set of people to help.

As for rewards, as we gradually move to a blueprint for blueprints model where operators actively review proposals for new function, I think we can establish a criteria for active contribution. Pointing out that a proposal will have scaling or upgrade issues saves hours of subsequent design, coding and deployment effort.


From: Anne Gentle [mailto:anne at openstack.org]
Sent: 31 March 2014 19:31
To: matt
Cc: openstack-operators at lists.openstack.org
Subject: Re: [Openstack-operators] Atlanta Summit - More Ops? ;)

On Mon, Mar 31, 2014 at 12:21 PM, matt <matt at nycresistor.com<mailto:matt at nycresistor.com>> wrote:
Incentives is a very interesting question.

What sort of incentives exist for devs?

Acknowledged as Contributor ( ATC ) to OpenStack -- ( resume fodder, free ticket to summit )
Voting in PTL, TC
I guess those are the official benefits.  There are the unofficial ones such as being saught after in the job market.
What sort of incentives do you envision for operators?
I mean with ATCs it's easy to set a barrier... did you contribute code.  How do you quantify an operators contribution to OpenStack?  When an operator is distinguished as an active contributor do we provide them with the same abilities as an ATC in dev?  There is no PTL for operators... no need for voting for TC... though maybe they should?
It raises interesting questions.  And I think it starts with, do we want to create a role for an ATC outside of development ( I think we already do for documentation contributors ).

Documentation contributors also use the git/gerrit method for updates, so it's easy to track ATC status. Same with the Security Group, they are now housing the Security Notes in a git repo under the openstack org held under the Documentation Program so that they can integrate with docs and get ATC status.

I think the Security team's model is a good one to investigate for operators, though I share Jay's concern about the combination of operator and admin. Still noodling on that one a little bit.

If so, are they an ATC or an ATC*.  Then comes the question of quantifying contribution.

I think we only have an ATC carrot at the moment. Would like to explore additional carrots but the governance in place is not easy to re-mold. It'd be great to get an ops team going that contributes to the Ops Guide and Admin Guides so work within the existing system while we continue to discuss possible additions to the governance.

In other words, I think that the incentives for engineers could be the same as for devs.

Great discussion all.


What are your thoughts?

On Mon, Mar 31, 2014 at 12:29 PM, Narayan Desai <narayan.desai at gmail.com<mailto:narayan.desai at gmail.com>> wrote:
Hi Matt.

I guess that I slice the problem a little bit differently. while I think that it would be useful to have an advocate on the TC, I think that would probably only fuel the culture mismatch.

I'm a big believer in incentives; I think in this case, all of the project level incentives are structured around development contributions, making unit tests pass, etc. There aren't any incentives explicitly built around reporting operational issues, helping devs get down to root cause, etc. I think that motivation from the top in this regard would help a lot more than having a ops advocate on the TC.

On Mon, Mar 31, 2014 at 10:26 AM, matt <matt at nycresistor.com<mailto:matt at nycresistor.com>> wrote:
Narayan I guess the first fundamental question, as I see it is, do operators deserve a seat at the TC table?
I can see the value of having that insight available at that level.  But, the same can be said of users, and security engineering / policy wonks.  And that opens a door to further pollution of what is supposed to be a fairly agile team intended to be focused on engineering rather than politics.

As with any large organization finding that happy medium for working with other departments is difficult.  I don't think that managerially we will find a total solution.  In fact, I think operators, users, and security folks should NOT be involved in the TC.  I think that the TC should be focused inwards on the engineering effort surrounding the development needs of OpenStack.  If we want to engage at a higher level, then a foundation seat I think makes more sense.
And, while I think it's important for operators, users, and security wonks to have buy in at the top of the organization helping to steer over all direction, I don't think that addresses what most of the operators really want.  Which is the ability to action change more directly.  The only analog I've seen that works for accomplishing this, is embeds.  What openstack needs is full fledged developers who are targeting operator needs.  We need operators who can code embedded in the standard development process. The reason we need this, is because any effort to contribute to OpenStack engineering, is going to require it.  There is just no way around that.  There's nothing wrong with that either.  This does work.
However, obviously not everyone gets to be able to contribute.  As stated previously, many of us have day jobs.  We're not given the same levity some of the community contributor show ponies are.  Someone has to go unclog the tubes and clean up after dazzling forays into user excess.  That's where I see an OSOG ( operators group ) coming into play.  The OSOG can set goals for the contributors or just contribute bugs / blueprints more effectively.  If operators want a seat at the table, I think the best way to do that is distill the operators needs in a closed group and then reach out into existing development methods as a unified front.  At that point, embedded operators who contribute can take the ball and run with it, and developers additionally can assist.
That's the model I think makes the most sense for us in the long term.  And I say this based on experience developing for openstack products and as an operator on two large openstack clouds.

On Mon, Mar 31, 2014 at 10:54 AM, Narayan Desai <narayan.desai at gmail.com<mailto:narayan.desai at gmail.com>> wrote:
I spend some time about a year ago writing up some thoughts about the user committee, and the major goals I thought it should have. The start of that thread is here:
but it continues into the January archives as well:

tl;dr I think that the most important problem for the user committee to solve is to figure out a dynamic where operators can fully participate in the project in a deep sense. Openstack has a very developer-centric ethos and structure, which clashes to a substantial extent with the operator community. We need to figure out how to productively marry ops culture (which doesn't focus on writing code, rather on building systems, figuring out how they break, what features they need, etc) with the development mission of the project.

I think that Tim's (& co) work on the user summit is a good step in this direction, but figuring out this culture issue is still the most important issue to solve.

I'm not lucky enough to have the gift of brevity; several of the mails in the thread above are quite long, and I think both describe the problems I see in detail, and also illustrate how the focus on developer culture doesn't necessarily get the project where it needs to go.

On Mon, Mar 31, 2014 at 9:35 AM, Jonathan Proulx <jon at jonproulx.com<mailto:jon at jonproulx.com>> wrote:
Some Monday morning pre-coffee thoughts

On Sun, Mar 30, 2014 at 10:15 PM, Tom Fifield <tom at openstack.org<mailto:tom at openstack.org>> wrote:

> So, my reading is we already have such governance established - but rather
> than being an individual, it is a committee - the user committee. We'll need
> to tweak it a bit I guess, but in fact it is already set up such that the TC
> _must_[1] listen to it ... for at least four hours per year ;)
That's definitely a front runner in my mind, cheers to all the hard
work the existing committee have done around surveys, the expanded
operations track and everything else.

I do think it's a bit confusingly named & that this stems from a
fundamental flaw in OpenStack community though, that there are two
parts of the community, "Developers" and "Users" and that "User" means
someone who deploys and maintains cloud infrastructure.

As I see it there's (at least) three major community segments, from
smallest to largest:

* Developer, who write the code
* Operators/Administrators/(pick your title), who build and maintain
production clouds
* Users who actually deploy applications on top of the cloud

Obviously many individuals and organizations fall into multiple
categories and within "Users" as writ above there's a variety of
constituencies that could be broken out.

In terms of Governance do the "User Committee" cover all that is not
dev?  That's really a huge amount of ground to cover and I do think
they've done a great job of it, especially on the ops side as
evidenced by this discussion, and I can see they're reaching out more
to the end users as well or starting to.

I'd be interested to hear what those who've been doing the job think
needs to be done to scale out and cover the whole constituency? More
member, more volunteer staff, sub-committees, distinct operators and
end user committees, or perhaps the existing structure is sufficient?

> Thoughts? What would you see this group doing?
I think the user surveys have been very valuable in seeing how
OpenStack is used in the wild, continuing that and refining the
questions so we can identify community priorities is a worthy goal and
an ongoing task that should definitely be continued.

Facilitating the organization of summit tracks & possible inter-summit
ops gatherings is another I think we have broad agreement on as that
seems to be happening.

Do we want to produce tangible best practices or example architectures
possibly by inviting in existing configuration management tools?  That
maybe a reach both in terms of our time availability and the interest
of the people who are doing that work now to come in under a new
umbrella.  If that, or something like that were our goal then a PTL
type structure would probably make more sense.


OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>

OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>

OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20140331/532849b5/attachment.html>

More information about the OpenStack-operators mailing list