Open Stack

Excerpts from John Dickinson's message of 2017-05-23 09:12:29 -0700:
> 
> I can sympathize with the "do it tomorrow" turns into 6 weeks later...
>
> Part of the issue for me, personally, is that a governance patch
> does *not* feel simple or lightweight. I assume (in part based on
> experience) that any governance patch I propose will be closely
> examined and I will be forced to justify every corner case and
> comment made. Frankly, writing the patch that will stand up too a
> critical eye will take a long time. I'll do it tomorrow...

Maybe this does point to a need to move that information somewhere else.
It would ultimately be the same people reviewing it, though. I feel
strongly that we need the review step, but if folks think a different
repository would make a difference I'd be happy to set that up.

> Let's take the py3 goal as an example. Note: I am *not* wanting
> to get into a discussion about particular py3 issues or whatever.
> This is a discussion on the goals process, and I'm only using one
> of the current goals as an example of why I haven't proposed a
> governance patch for it.

> Swift does not support Py3. So clearly, there's work to be done
> to meet the goal. I've talked with others in the community about
> some of the blockers and concerns about porting to Py3. Several of
> the concerns are not trivial and will take substantial work to
> overcome[1]. A governance patch will need to list these issues, but
> I don't know if this is a complete list. If I propose a list that's
> incomplete, I feel like I'll be judged on the list I first proposed
> ("you finished the list, why doesn't it work?") instead of being a
> more dynamic process. I need to spend more time understanding what
> the issues are to make sure I have a complete list. I'll propose
> that patch tomorrow...

The patch does not necessarily need to list every detail. The purpose
of having a list of artifacts in the goal document is so that anyone
who wants to understand the state of the implementation can go look
there.  So, for example, if you're using a wiki page or an etherpad
to keep track of the details within the team, the patch only needs
to include a link to that. Some teams have done more, linking to
specs or changes that are already under review. Exactly what type
of artifact counts for a team is really up to that team.

The point is to show that each team is aware of the goal, and that
they've put together information in a place that someone outside
of the team can find it to either help, or at least follow progress.

> The outstanding work to get Py3 support in Swift is very large.
> Yet there are more goals being discussed now, and there's no way I
> can get Py3 support in Swift in Pike. Or Queens. Or probably Rocky
> either. That's not to say it isn't an important goal, but the scope
> combined with the TC deadline means that my governance patch for
> this goal (the tl;dr version is "not gonna happen") has to address
> this in sufficient detail to stand up to review by TC members who
> are on the PSF! I guess I'll start writing that tomorrow...

Some teams have a bit of a head start, but we expect many teams to
find the Python 3 work more than can be completed in a cycle. That's
perfectly OK. At the end of the cycle, we'll see where things stand,
and determine what the next steps are. That retrospective process
will be up to the teams, but I would expect it to factor into the
TC's decisions about what goals are adopted for Queens.

We don't want to have a big pile of unmet goals that all teams are
struggling to make progress on. That's why we have been limiting
ourselves to 1-2 goals per cycle.

> While I know that Py3 support is important, I also have to
> prioritize it against other important things. My employer has
> prioritized certain features because that directly impacts our
> ability to add customers (which directly affects my ability to get
> paid). Other employers in the community are doing the same for their
> employees. In the broader community, as clusters have grown over

There is undoubtedly tension between upstream and downstream needs
in some of these areas. We see that tension a lot with cross-project
initiatives. I don't have a good generic answer to the problem of
balancing community and employer needs, so I think the conversation
will have to happen case-by-case.

If we're finding that all of the contributors to a team are discouraged
from working on technical debt issues or other community goals in,
we'll need to address that. Uncovering that bit of information would
be an important outcome for the goals process, especially if it is
stated as directly as "no team member is being given time by their
employer to work on this community goal." If there is no response
from a team at all, though, we have no idea why that is the case.

If we know a team has issues tracking the goals due to a lack of
resources, then when the Board asks "how can we help," as they do
every time we have a joint meeting, one answer might be to ask
companies to give folks some time to work on these things. Or maybe
it shows that the community direction is completely out of line
with where the market is heading, which is also valuable feedback.

> the years, certain original design decisions are showing their age
> and actively causing operator pain. Perhaps we can survive for a
> while longer without solving these (we managed to ignore these
> issues for the last five years), but as time passes these pain
> points actually become reasons to use a different storage system
> other than Swift. We can't ignore stuff like container sharding and
> improving capacity adjustments any longer. Furthermore, we've got
> plans underway (in part to solve some of the aforementioned issues)
> to replace Python code with Golang code. That makes a Py3 port that
> much easier, but it's work that competes with the porting work in
> other parts of the codebase. Add in to all this that I've talked
> to exactly zero people who could not use Swift because it's written
> in Py2, and the prioritization of the Py3 port gets very easy. But
> writing a governance patch to say that the Py3 work can't be
> prioritized is confrontational. So let's put it off until tomorrow...

You didn't want to get into a discussion of the merits of any
specific arguments for or against the idea of porting, so I won't
address this part in this thread.

I will say that if the Swift team's response to the Python 3 goal
is that there is no plan to port to Python 3 because Go is the
target for a language with long-term support, from my perspective
that might be OK.  I don't think that answer works for every team,
but the technical case for some use of golang has already been made
for Swift and I could see that being a viable alternative migration
path that meets the overarching goal of getting off of what will
soon be an unsupported language runtime. That particular response
is likely to trigger more discussion, of course, than a simple
reference to tracking artifacts more directly in line with the goal,
but I could see myself supporting the approach in this case.

> There's been work in the past to port Swift to Py3. I'm grateful
> to all those people who have spent the time and effort to move Swift
> closer to Py3 support. Py3 port patches are *not* simple to review,
> and we've had at lest 2 or 3 significant regressions that were
> introduced because of porting work. The Py3 port patches are hard
> to review, and after reviewing a few multi-thousand line patches
> that appear to mostly be syntactic but might in fact be hiding a
> subtle error, it's hard to maintain the motivation to pick up Py3
> port patches over other important patches to review[2]. Knowing
> that the Py3 patches are hard for the community to review, it's
> hard to estimate how long the porting process will take and even
> harder to commit to the completion dates the TC set for them. There's
> progress being made, but it's really slow. I'll put off proposing
> the governance patch until we can show some better progress. I'll
> propose the patch tomorrow...

Yep, it sounds like those patches should probably have been broken
up differently. I didn't review them, so I can't say for certain.
Different teams are going to have different preferences on the
structure of the changes, so it's not something I think we considered
trying to define up front.

> And here we are today. Six weeks after the deadline for getting
> the initial patch in. The Py3 goal is only used as an example, but
> I think there are some fundamental systemic issues with the goals
> process and community, as Thierry suggested. I don't know what
> exactly they are, much less what the answers are, but I submit my
> personal experience as evidence in the open.

Thanks for writing this up, John. I know there has been some friction in
the past, and I think having this conversation is a step in the right
direction to fixing that.

> I should propose that governance patch soon. Maybe tomorrow...

Please do. We want this process to be a shared conversation, but
by definition it won't be if people aren't participating. And thank you
again for this email.

Doug