[openstack-dev] [fuel][puppet] The state of collaboration: 5 weeks

Matt Fischer matt at mattfischer.com
Mon Jul 20 18:24:13 UTC 2015

Dmitry and I chatted some this morning and I'd at least like to address
those issues so that we can resolve them and move on to the other
discussions. I am not speaking for Emilien here, just jumping in as a core
and trying to help resolve some of this.

As Emilien notes, many of us do not work full-time on puppet code, so
sometimes reviews can slip. IRC messages can also be missed especially
during summer months with many of us on vacations. For that reason, Dmitry
and the Fuel team will be bringing reviews to the weekly meeting to call
out ones that need or are overdue for attention. I believe that missed,
overlooked, ignored, or forgotten reviews are not a new issue for an
OpenStack project, but hopefully this solution will work to improve
throughput. This solution applies to anyone who is possibly not getting
enough attention on the review.

Our review dashboard is here: https://goo.gl/bSYJt8

Our weekly meeting schedule link is posted by Monday morning and the
meeting takes place Tuesday. Anyone can add to the schedule.

I also agree with both Emilien and Dmitry that I've seen Fuel contributions
increase since the initial conversation here and I hope to see that trend
continue. It will be better for everyone if we can collaborate well.

On Sat, Jul 18, 2015 at 4:32 PM, Dmitry Borodaenko <dborodaenko at mirantis.com
> wrote:

> It has been 5 weeks since Emilien has asked Fuel developers to contribute
> more
> actively to Puppet OpenStack project [0]. We had a lively discussion on
> openstack-dev, myself and other Fuel developers proposed some concrete
> steps
> that could be done to reconcile the two projects, the whole thread was
> reported
> and discussed on Linux Weekly News [1], and the things were looking up.
> [0]
> http://lists.openstack.org/pipermail/openstack-dev/2015-June/066544.html
> [1] https://lwn.net/Articles/648331/
> And now, 5 weeks later, Emilien has reported to the Technical Committee
> that
> there has been no progress on reconciliation between Fuel and Puppet
> OpenStack,
> and used his authority as a PTL to request that the Fuel's proposal to join
> OpenStack [2] is rejected.
> [2] https://review.openstack.org/199232
> In further comments on the same review, Emilien has claimed that there's
> "clearly less" contribution to Puppet OpenStack from Fuel developers than
> before, and even brought up an example of a review in puppet-horizon that
> was
> proposed and then abandoned by Fuel team [3]. Jay went as far as calling
> that
> example "an obvious failure of working with the upstream Puppet-OpenStack
> community".
> [3] https://review.openstack.org/198119
> Needless to say, I found all these claims deeply disturbing, and had to
> look
> closely into what's going on.
> The case of the puppet-horizon commit has turned out to be surprisingly
> obvious.
> Even before looking into the review comments, I could see a technical
> reason
> for abandoning the commit: if there is a bug in a component, fixing that
> bug in
> the package is preferrable to fixing it in puppet, because it allows
> anybody to
> benefit from the fix, not just the people deploying that package with
> puppet.
> And if you do look at the review in question, you'll find that immediately
> (14
> minutes, and that at 6pm on Friday night!) after Jay has asked in the
> comments
> to the review why it was abandoned, the commit author from the Fuel team
> has
> explained that this patch was a workaround for a packaging problem, and
> that
> this was pointed out in the review by a Horizon core reviewer more than a
> week
> ago, and later corroborated by a Puppet OpenStack core reviewer. Further
> confirming that fixing this in the package instead of in puppet-horizon
> was an
> example of Fuel developers agreeing with other Puppet OpenStack
> contributors
> and doing the right thing.
> Emilien has clearly found this case important enough to report to the TC,
> and
> yet didn't find it important enough to simply ask Fuel developers why they
> chose to abandon the commit. I guess you can call that an obvious failure
> to
> work together.
> Looking further into Fuel team's reviews for puppet-horizon, I found
> another
> equally disturbing example [4].
> [4] https://review.openstack.org/190548
> Here's what I see in this review:
> a) Fuel team has spent more than a month (since June 11) on trying to land
> this
> patch.
> b) 2 out of 5 negative reviews are from Fuel team, proving that Fuel
> developers
> are not "rubberstamping" each other's commits as was implied by Emilien's
> comments on the TC review above.
> c) There was one patch set that was pushed 3 business days after a negative
> review, all other patch sets (11 total) were pushed no later than next day
> after a negative review.
> All in all, I see great commitment and effort on behalf of Fuel team,
> eventually awarded by a +2 from Michael Chapman.
> On the same day (June 30), Emilien votes -2 for a correction in a comment,
> and
> walks away from the review for good. 18 days and 4 patch sets and 2
> outstanding
> +1's later, the review remains blocked by that -2. Reaching out on
> #puppet-openstack [5] didn't help, either.
> [5] http://irclog.perlgeek.de/puppet-openstack/2015-07-08#i_10867124
> At the same time, Emilien has commented on the TC review that the only
> metric
> he considers reflective of Fuel's contribution to Puppet OpenStack is
> merged
> commits. Isn't that a Catch-22 situation, requesting more merged commits
> and
> refusing to merge them?
> When I compare the trends in the number of patch sets pushed by Fuel
> developers
> [6] against the number of merged commits [7], I see the same picture. It's
> really obvious when comparing both graphs side by side.
> [6]
> http://stackalytics.com/?module=puppetopenstack-group&metric=patches&company=mirantis
> [7]
> http://stackalytics.com/?module=puppetopenstack-group&metric=commits&company=mirantis
> Between May and July, the peak number of patchsets in Puppet OpenStack
> from all
> contributors increased by a factor of 1.78, while patchsets associated with
> Mirantis jumped up by a factor of 6.75. At the same time, merged commits
> from
> all contributors increased by 2.44 (meaning 30% less patch sets per commit
> than
> in May), for Mirantis the factor is only 2.0 (meaning 3 times as many patch
> sets per commit as in May).
> Just look at the attached picture to see how obviously massive is the
> increase
> in the effort that Fuel team contributes to Puppet OpenStack. One can
> argue for
> different ways to explain why this effort does not convert into merged
> commits,
> but based on two examples above I find it very hard to believe that it can
> be
> explained by a lack of trying.
> In fact, this is exactly the situation that I have predicted in the
> original
> thread when I said that commmitment to collaborate has to be mutual [8].
> [8]
> http://lists.openstack.org/pipermail/openstack-dev/2015-June/066731.html
> Fuel team can commit to reporting bugs and posting patches, Fuel team can
> spend
> reasonable amount of time and effort to respond to review comments, but all
> that is worth nothing if there's no commitment from Puppet OpenStack team
> to
> review and merge these patches in a timely manner.
> Looking through the collaboration thread, the only commitment I see is
> Emilien's promise that "if you submit a good patch now, it will land in
> maximum
> one week or so", and his comment on the TC review: "The end of the thread
> was
> kind of consensus to me and I was pretty happy to see Dmitry was really
> open to
> the discussion".
> Clearly this kind of consensus is not working out, and I think at least one
> reason is that my proposal for Puppet OpenStack developers to take over
> stalled
> commits was rejected. One way or another, cases like "16 days since the
> first
> patch set that had no negative reviews" that obviously violate the "land
> within
> a week" commitment have to be identified and prioritized. Being on the
> hook to
> take them over if they don't land on time is one way to introduce a
> motivation
> to make that happen. There could be other ways to address it, for example
> having a review dashboard that highlights stuck reviews (as I proposed in
> the
> same email) would help add some visibility into the problem.
> Please don't get me wrong. In the same reviews I've highlighted above I see
> excellent example of collaboration from Puppet OpenStack team, insightful
> and
> timely comments, and I'm not denying the effort the team spends on
> onboarding
> new contributors flooding in from Fuel. But I don't see how this situation
> can
> be called "no progress" and "reduced contribution from Fuel" when making
> further progress is limited by Puppet OpenStack team's ability to review
> incoming patches much more than by Fuel team's commitment to proposing and
> updating them.
> --
> Dmitry Borodaenko
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150720/eb406ed7/attachment.html>

More information about the OpenStack-dev mailing list