Open Stack

On 06/12/2015 05:43 AM, Dmitry Borodaenko wrote:
> On Thu, Jun 11, 2015 at 11:43:09PM -0400, Emilien Macchi wrote:
>> What about code history and respect of commit ownership?
>> I'm personally wondering if it's fair to copy/paste several thousands of
>> lines of code from another Open-Source project without asking to the
>> community or notifying the authors before. I know it's Open-Source and
>> Apache 2.0 but well... :-)
> 
> Being able to copy code without having to ask for permission is exactly
> what Free Software (and more recently, Open Source) is for.
> 
> You can't rely on commit history and even changelog to track attribution
> and licensing, source tree itself should contain all appropriate
> copyright notices and licenses, and we keep all of those intact:
> 
> https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/cinder/Modulefile#L3-L4
> https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/cinder/LICENSE
> 
> Besides, there's a historic precedent that stripping commit history is
> acceptable even with GPL:
> 
> https://lwn.net/Articles/432012/

Oh I'm pretty sure you don't violate any law. I'm just not sure it was
the right way to use the Puppet modules.

I don't think there is value to discuss more about this 'copy/paste'
thing, let's move on to interesting stuffs.

> 
>>>> Should not it be the way around?
>>>> Puppet OpenStack modules provide the original code. If there is a bug,
>>>> it has to be fixed in the modules. Puppet OpenStack developers don't
>>>> have time/bandwidth and moreover don't want to periodically have a
>>>> look at Fuel git history. I'm not sure this is the best solution for
>>>> the community.
>>> (...)
>>>> The reality (and again I won't blame any patch, you can find them on
>>>> Gerrit) is that most of patches are not merged and in staled status.
>>>> If I can suggest something, the policy should be more "upstream first"
>>>> where you submit a patch upstream, you backport it downstream, and in
>>>> the until it's merged you should make sure it land upstream after
>>>> community review process. The last step is I think the problem I'm
>>>> mentioning here and part of the root cause of this topic.
>>>
>>> Yes, this right here is the biggest point of contention in the whole
>>> discussion.
>>>
>>> The most problematic implication of what you're asking for is the
>>> additional effort that it would require from Fuel developers. When you
>>> say that Puppet OpenStack developers don't have time to look at Fuel git
>>> history for bugfixes, and then observe that actually Fuel developers do
>>> propose their patches to upstream, but those patches are stalled in the
>>> community review process, this indicates that you don't consider taking
>>> over and landing these patches a priority:
>>
>> We don't consider taking the patches?
> 
> Why do you misinterpret my words this way here, and then few paragraphs
> later you demonstrate that you clearly understand the difference between
> "taking patches" and "taking over patches"?
> 
>> Please go on Gerrit, look at the patches and tell me if there is no
>> review from Puppet OpenStack community. Most of the patchs are -1 or
>> not passing unit testing which means the code can't be merged.
>>
>> Let me give you examples so you can see Puppet OpenStack folks is doing
>> reviews on patchs from Fuel team:
>> https://review.openstack.org/#/c/170485/
>> https://review.openstack.org/#/c/157004/
>> https://review.openstack.org/#/c/176924/
>> https://review.openstack.org/#/c/168848/
>> https://review.openstack.org/#/c/130937/
>> https://review.openstack.org/#/c/131710/
>> https://review.openstack.org/#/c/174811/
>>
>> And this is only 'in progress' patches. A lot of fixed have been
>> abandoned upstream. You can easily query them on Gerrit.
> 
> Once again, this only disproves your concern that Puppet OpenStack
> developers would have to waste time digging through Fuel commit history
> to track down bugfixes. Fuel developers are taking care of this for you.

Good to know so we have a first action:
"Fuel developers to track down every bug in Puppet modules that are
fixed in Fuel, at least by submitting a patch upstream and making sure
the review stays alive. If the developer can't spend more time on a
pending patch, the developer can gently ask on the ML or IRC if someone
from Puppet OpenStack group can takeover to finish the code, and use
Co-Authored-By".

>>> The fact that you have started this thread means that you actually do
>>> care to get these bugfixes into Puppet OpenStack. If that's true, maybe
>>> you can consider a middleground approach: Fuel team agrees to propose
>>> all our changes to upstream (i.e. do a better job at something we've
>>> already committed to unilaterally), and you help us land the patches we
>>> propose, and take over those that get stalled when the submitter from
>>> Fuel team has moved on to other tasks?
>>
>> If I understand correctly, you're asking for Puppet OpenStack group to
>> take over patches that are sent from Fuel team but have negative reviews
>> (not passing unit tests, not compliant with Puppet best practices), just
>> because they have to switch to another task and they can't take time to
>> finish the upstream work?
> 
> Yes, except for the judgemental parts of your wording:

I sincerely apologize if you think I gave a judgment. I honestly did not
but as you may already know, I'm not native english speaker. It can be
hard for me to expose my points like it would be in my native language.

> 
> I'm asking for Puppet OpenStack developers *to consider it an option* to
> take over patches (including those that are sent from Fuel team) blocked
> by negative reviews (even for objective reasons such as failing unit
> tests), when the original patch submitter has decided that landing that
> patch in upstream is no longer as high in their priorities as other
> activities.

The option is valid *only* if there is a communication between both
groups, which is not the case today and which is my goal in this thread.

> I fully understand that this would require more time from Puppet
> OpenStack developers, but I suspect that it would take less time for
> them to land such a patch than it would for a non-core such as a Fuel
> developer. In all but most trivial cases, it's even going to be less
> time than it took that Fuel developer to find the bug, analyze the root
> cause, and produce and test the first working implementation of the fix.

+1, it could be a benefit for both groups if this collaboration works.

> If all these assumptions are true, wouldn't it be a worthwhile
> investment of Puppet OpenStack developers' time, and a fair trade-off
> for the value added to upstream manifests by all those patches?
> 
>> This is definitely not how OpenStack works.
> 
> Even if you were right and taking over other people's commits weren't an
> acceptable practice in OpenStack, you're still committing the Appeal to
> Authority fallacy. OpenStack isn't perfect (no project ever is), and
> even more so isn't the golden standard for fostering collaboration.

Taking over people's commits is something that happen every day and we
are happy with that. Though they happen after a communication between
the developers, so I don't see any trouble here.
Again, communication between both groups will really be a good start.

>> Let me propose you another middleground approach:
>> Fuel team agrees to propose all our changes to upstream (i.e. do a
>> better job at something we've already committed to unilaterally), and
>> Puppet Openstack group help Fuel team  to land the patches they propose,
>> in helping Fuel team to be on-boarded in our community (my goal of this
>> thread by the way).
>>
>> On-boarding means:
>> * be on #puppet-openstack and participate (discussions about patches,
>> bugs, blueprints, etc)
>> * participate to design sessions at Summit
>> * partitipate to mailing list
>> * participate to weekly meetings
>> * understand our code best practice, like OpenStack does have conventions.
> 
> All reasonable, but one-sided. You ask Fuel team to commit to a concrete
> list of time-consuming activities in exchange for a vague offer of help.
> One concrete way to help would have been to take over stalled patches,
> there may be others. But if even that is too much work, what else do you
> have in mind when you promise to help?

Again you're asking for Puppet OpenStack group to take over stalled
patches because it's too 'time-consuming' for Fuel devs for some patches.

Do you honestly Puppet OpenStack groups have this bandwidth?
Our group is not that big, much smaller than whole Fuel team doing
Puppet. What you asking is just impossible.

What I propose again is more involvement from Fuel team in their
upstream patches and understand the Puppet modules best practices so
their patches can merge quicker. What I'm waiting from Puppet OpenStack
group, is that we continue to do reviews and help Fuel team to land
their patches like any other patches.
I would not do differentiation between a patch from a Fuel dev or not
though.
If a Fuel dev gently ask on #puppet-openstack about taking over a patch
or asking more help. I'm sure our group will be really happy to help*.

Define help:
* Giving technical feedback on a patch using Gerrit
* If needed, giving more insights by IRC or ML
* Worst case: after discussion between both groups, take over a patch,
and give feedback to the Fuel dev at the end about the upstream merged
patch.

Puppet OpenStack project works (or try to) like any other OpenStack project.

>>> A better alternative would be to make all upstream Puppet OpenStack
>>> directly usable in Fuel, but even if we figure out a way to make that
>>> work, it will take a long journey to get there. On the upstream side,
>>> Fuel core reviewers would have to also be upstream core reviewers, and
>>> Fuel CI would have to be allowed to vote on upstream commits. On Fuel
>>> side, we'd have to complete the reconciliation and modularization of all
>>> our forked modules, and move all Fuel CI to openstack-infra. The
>>> potential benefits for both sides are tremendous, but only after we
>>> essentially merge the two projects. Even if that's achievable, is that
>>> something whole Puppet OpenStack community is interested in?
>>
>> * Having people from Fuel team in Puppet OpenStack core team would be
>> done by meritocracy as any developer in the community actively contributing.
> 
> Which is exactly why I said that it will take a long journey. And until
> that happens and we can be reasonably sure that landing a patch in
> upstream is just as easy and fast as landing it in Fuel, contributing to
> Fuel will remain the priority for Fuel developers, and we'll have to
> continue to merge patches to Fuel before they get +2 from Puppet
> OpenStack core reviewers. Seems to be a chicken and egg problem, and I
> don't see a good solution so far.
> 
>> * Having third party CI: we already have TripleO jobs that are run for
>> every commit in Puppet Modules.
> 
> TripleO is not going to be enough, we'll need Fuel CI, and we have long
> wanted to make it public.

TripleO CI is the only one installer that gates our modules for now,
which is to me a very good start for third party CI.

> 
>> Though you'll have to make your CI 100% public, which is not the case
>> today. Example on https://review.fuel-infra.org/ with
>> https://review.fuel-infra.org/#/c/7568/ or any patch, if I want to
>> read the logs of CI jobs, I can't access to your CI servers:
>> http://osci-jenkins.srt.mirantis.net:8080/job/7.0.mos-new.build-deb-request/381/
>> So you would have to expose your servers so the community can debug
>> the failures.
> 
> This is a bug, not a feature, of our engineering process. We're working
> on it, and it's also an area where we could use some help. I wasn't in
> Vancouver in person, but I've got the impression that our offer to
> expose Fuel CI to the community has received a lukewarm reception from
> openstack-infra at best.
> 

FWIW, TripleO CI is non-voting, and does not disturb our workflow. If
one day you use our modules correctly, I see a huge interest to gate our
modules with Fuel, but indeed, it requires more discussion which is not
related to this thread.
My thought on this is, the best we cover, better our modules are good,
awesome our products will be downstream.
-- 
Emilien Macchi

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150612/7133a562/attachment.pgp>