Open Stack

On Thu, Jun 11, 2015 at 11:43:09PM -0400, Emilien Macchi wrote:
> What about code history and respect of commit ownership?
> I'm personally wondering if it's fair to copy/paste several thousands of
> lines of code from another Open-Source project without asking to the
> community or notifying the authors before. I know it's Open-Source and
> Apache 2.0 but well... :-)

Being able to copy code without having to ask for permission is exactly
what Free Software (and more recently, Open Source) is for.

You can't rely on commit history and even changelog to track attribution
and licensing, source tree itself should contain all appropriate
copyright notices and licenses, and we keep all of those intact:

https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/cinder/Modulefile#L3-L4
https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/cinder/LICENSE

Besides, there's a historic precedent that stripping commit history is
acceptable even with GPL:

https://lwn.net/Articles/432012/

> >> Should not it be the way around?
> >> Puppet OpenStack modules provide the original code. If there is a bug,
> >> it has to be fixed in the modules. Puppet OpenStack developers don't
> >> have time/bandwidth and moreover don't want to periodically have a
> >> look at Fuel git history. I'm not sure this is the best solution for
> >> the community.
> > (...)
> >> The reality (and again I won't blame any patch, you can find them on
> >> Gerrit) is that most of patches are not merged and in staled status.
> >> If I can suggest something, the policy should be more "upstream first"
> >> where you submit a patch upstream, you backport it downstream, and in
> >> the until it's merged you should make sure it land upstream after
> >> community review process. The last step is I think the problem I'm
> >> mentioning here and part of the root cause of this topic.
> > 
> > Yes, this right here is the biggest point of contention in the whole
> > discussion.
> > 
> > The most problematic implication of what you're asking for is the
> > additional effort that it would require from Fuel developers. When you
> > say that Puppet OpenStack developers don't have time to look at Fuel git
> > history for bugfixes, and then observe that actually Fuel developers do
> > propose their patches to upstream, but those patches are stalled in the
> > community review process, this indicates that you don't consider taking
> > over and landing these patches a priority:
> 
> We don't consider taking the patches?

Why do you misinterpret my words this way here, and then few paragraphs
later you demonstrate that you clearly understand the difference between
"taking patches" and "taking over patches"?

> Please go on Gerrit, look at the patches and tell me if there is no
> review from Puppet OpenStack community. Most of the patchs are -1 or
> not passing unit testing which means the code can't be merged.
> 
> Let me give you examples so you can see Puppet OpenStack folks is doing
> reviews on patchs from Fuel team:
> https://review.openstack.org/#/c/170485/
> https://review.openstack.org/#/c/157004/
> https://review.openstack.org/#/c/176924/
> https://review.openstack.org/#/c/168848/
> https://review.openstack.org/#/c/130937/
> https://review.openstack.org/#/c/131710/
> https://review.openstack.org/#/c/174811/
> 
> And this is only 'in progress' patches. A lot of fixed have been
> abandoned upstream. You can easily query them on Gerrit.

Once again, this only disproves your concern that Puppet OpenStack
developers would have to waste time digging through Fuel commit history
to track down bugfixes. Fuel developers are taking care of this for you.

> > The fact that you have started this thread means that you actually do
> > care to get these bugfixes into Puppet OpenStack. If that's true, maybe
> > you can consider a middleground approach: Fuel team agrees to propose
> > all our changes to upstream (i.e. do a better job at something we've
> > already committed to unilaterally), and you help us land the patches we
> > propose, and take over those that get stalled when the submitter from
> > Fuel team has moved on to other tasks?
> 
> If I understand correctly, you're asking for Puppet OpenStack group to
> take over patches that are sent from Fuel team but have negative reviews
> (not passing unit tests, not compliant with Puppet best practices), just
> because they have to switch to another task and they can't take time to
> finish the upstream work?

Yes, except for the judgemental parts of your wording:

I'm asking for Puppet OpenStack developers *to consider it an option* to
take over patches (including those that are sent from Fuel team) blocked
by negative reviews (even for objective reasons such as failing unit
tests), when the original patch submitter has decided that landing that
patch in upstream is no longer as high in their priorities as other
activities.

I fully understand that this would require more time from Puppet
OpenStack developers, but I suspect that it would take less time for
them to land such a patch than it would for a non-core such as a Fuel
developer. In all but most trivial cases, it's even going to be less
time than it took that Fuel developer to find the bug, analyze the root
cause, and produce and test the first working implementation of the fix.

If all these assumptions are true, wouldn't it be a worthwhile
investment of Puppet OpenStack developers' time, and a fair trade-off
for the value added to upstream manifests by all those patches?

> This is definitely not how OpenStack works.

Even if you were right and taking over other people's commits weren't an
acceptable practice in OpenStack, you're still committing the Appeal to
Authority fallacy. OpenStack isn't perfect (no project ever is), and
even more so isn't the golden standard for fostering collaboration.

> Let me propose you another middleground approach:
> Fuel team agrees to propose all our changes to upstream (i.e. do a
> better job at something we've already committed to unilaterally), and
> Puppet Openstack group help Fuel team  to land the patches they propose,
> in helping Fuel team to be on-boarded in our community (my goal of this
> thread by the way).
> 
> On-boarding means:
> * be on #puppet-openstack and participate (discussions about patches,
> bugs, blueprints, etc)
> * participate to design sessions at Summit
> * partitipate to mailing list
> * participate to weekly meetings
> * understand our code best practice, like OpenStack does have conventions.

All reasonable, but one-sided. You ask Fuel team to commit to a concrete
list of time-consuming activities in exchange for a vague offer of help.
One concrete way to help would have been to take over stalled patches,
there may be others. But if even that is too much work, what else do you
have in mind when you promise to help?

> > A better alternative would be to make all upstream Puppet OpenStack
> > directly usable in Fuel, but even if we figure out a way to make that
> > work, it will take a long journey to get there. On the upstream side,
> > Fuel core reviewers would have to also be upstream core reviewers, and
> > Fuel CI would have to be allowed to vote on upstream commits. On Fuel
> > side, we'd have to complete the reconciliation and modularization of all
> > our forked modules, and move all Fuel CI to openstack-infra. The
> > potential benefits for both sides are tremendous, but only after we
> > essentially merge the two projects. Even if that's achievable, is that
> > something whole Puppet OpenStack community is interested in?
> 
> * Having people from Fuel team in Puppet OpenStack core team would be
> done by meritocracy as any developer in the community actively contributing.

Which is exactly why I said that it will take a long journey. And until
that happens and we can be reasonably sure that landing a patch in
upstream is just as easy and fast as landing it in Fuel, contributing to
Fuel will remain the priority for Fuel developers, and we'll have to
continue to merge patches to Fuel before they get +2 from Puppet
OpenStack core reviewers. Seems to be a chicken and egg problem, and I
don't see a good solution so far.

> * Having third party CI: we already have TripleO jobs that are run for
> every commit in Puppet Modules.

TripleO is not going to be enough, we'll need Fuel CI, and we have long
wanted to make it public.

> Though you'll have to make your CI 100% public, which is not the case
> today. Example on https://review.fuel-infra.org/ with
> https://review.fuel-infra.org/#/c/7568/ or any patch, if I want to
> read the logs of CI jobs, I can't access to your CI servers:
> http://osci-jenkins.srt.mirantis.net:8080/job/7.0.mos-new.build-deb-request/381/
> So you would have to expose your servers so the community can debug
> the failures.

This is a bug, not a feature, of our engineering process. We're working
on it, and it's also an area where we could use some help. I wasn't in
Vancouver in person, but I've got the impression that our offer to
expose Fuel CI to the community has received a lukewarm reception from
openstack-infra at best.

-- 
Dmitry Borodaenko