Open Stack

On 06/11/2015 10:31 PM, Dmitry Borodaenko wrote:
> On Thu, Jun 11, 2015 at 05:39:28PM -0400, Emilien Macchi wrote:
>> On 06/11/2015 10:36 AM, Matthew Mosesohn wrote:
>>> I'm not saying it's the most community-oriented approach, but Fuel
>>> would have never evolved and matured without it. The attribution in
>>> commits is lost because our directory namespace differs such that it
>>> can't just be merged cleanly. Revisiting submodules is an option,
>>> but it led to maintenance problems in the past.
>> What kind of problems?
> 
> It was before I got involved with Fuel, but I can offer a guess.
> Managing submodules introduces operational overhead and with it more
> opportunities for failures and mishaps than a single flat git repo.
> 
> Flat repo makes it more difficult to reconcile with upstream modules,
> but, when using the process I have described in my previous email to
> this thread, not as difficult as one could think. We also reconcile an
> average module with upstream much less frequently (no more than once per
> Fuel release) than we make commits to that module (many times per
> release), which also tilts the overhead balance if favor of using a flat
> repo.

What about code history and respect of commit ownership?
I'm personally wondering if it's fair to copy/paste several thousands of
lines of code from another Open-Source project without asking to the
community or notifying the authors before. I know it's Open-Source and
Apache 2.0 but well... :-)

>> You also could have used forks of modules, applied your patches and done
>> rebase from time to time when you like.
>> Using a Puppetfile in your installer and you're all set.
> 
> We do the "apply patches and rebase from time to time" part in a single
> repo, operating on a subdirectly within it doesn't actually add any
> overhead to this part of the workflow, as long as we keep track of sync
> commits properly.
> 
>> The "maintenance problems" thing does not sound right to me here, I
>> think it's more expensive to maintain files than git repositories.
> 
> To sum up, I don't think the difference is that great, or impact of
> doing it one way or the other that important. The current layout works
> well enough for Fuel team, and as you said yourself, developers of
> upstream modules are not likely to pro-actively harvest Fuel for
> bugfixes even if we change our repository structure.
> 
>>> The difference is that Fuel is being worked on in the open in
>>> StackForge. Anyone is free to contribute to Fuel as he or she
>>> wishes, take our patches, or review changesets.
>> Should not it be the way around?
>> Puppet OpenStack modules provide the original code. If there is a bug,
>> it has to be fixed in the modules. Puppet OpenStack developers don't
>> have time/bandwidth and moreover don't want to periodically have a
>> look at Fuel git history. I'm not sure this is the best solution for
>> the community.
> (...)
>> The reality (and again I won't blame any patch, you can find them on
>> Gerrit) is that most of patches are not merged and in staled status.
>> If I can suggest something, the policy should be more "upstream first"
>> where you submit a patch upstream, you backport it downstream, and in
>> the until it's merged you should make sure it land upstream after
>> community review process. The last step is I think the problem I'm
>> mentioning here and part of the root cause of this topic.
> 
> Yes, this right here is the biggest point of contention in the whole
> discussion.
> 
> The most problematic implication of what you're asking for is the
> additional effort that it would require from Fuel developers. When you
> say that Puppet OpenStack developers don't have time to look at Fuel git
> history for bugfixes, and then observe that actually Fuel developers do
> propose their patches to upstream, but those patches are stalled in the
> community review process, this indicates that you don't consider taking
> over and landing these patches a priority:

We don't consider taking the patches? Please go on Gerrit, look at the
patches and tell me if there is no review from Puppet OpenStack
community. Most of the patchs are -1 or not passing unit testing which
means the code can't be merged.

Let me give you examples so you can see Puppet OpenStack folks is doing
reviews on patchs from Fuel team:
https://review.openstack.org/#/c/170485/
https://review.openstack.org/#/c/157004/
https://review.openstack.org/#/c/176924/
https://review.openstack.org/#/c/168848/
https://review.openstack.org/#/c/130937/
https://review.openstack.org/#/c/131710/
https://review.openstack.org/#/c/174811/

And this is only 'in progress' patches. A lot of fixed have been
abandoned upstream. You can easily query them on Gerrit.
> 
> http://lifehacker.com/5892948/instead-of-saying-i-dont-have-time-say-its-not-a-priority
> 
> The fact that you have started this thread means that you actually do
> care to get these bugfixes into Puppet OpenStack. If that's true, maybe
> you can consider a middleground approach: Fuel team agrees to propose
> all our changes to upstream (i.e. do a better job at something we've
> already committed to unilaterally), and you help us land the patches we
> propose, and take over those that get stalled when the submitter from
> Fuel team has moved on to other tasks?

If I understand correctly, you're asking for Puppet OpenStack group to
take over patches that are sent from Fuel team but have negative reviews
(not passing unit tests, not compliant with Puppet best practices), just
because they have to switch to another task and they can't take time to
finish the upstream work?

This is definitely not how OpenStack works.

Let me propose you another middleground approach:
Fuel team agrees to propose all our changes to upstream (i.e. do a
better job at something we've already committed to unilaterally), and
Puppet Openstack group help Fuel team  to land the patches they propose,
in helping Fuel team to be on-boarded in our community (my goal of this
thread by the way).

On-boarding means:
* be on #puppet-openstack and participate (discussions about patches,
bugs, blueprints, etc)
* participate to design sessions at Summit
* partitipate to mailing list
* participate to weekly meetings
* understand our code best practice, like OpenStack does have conventions.

> 
> A better alternative would be to make all upstream Puppet OpenStack
> directly usable in Fuel, but even if we figure out a way to make that
> work, it will take a long journey to get there. On the upstream side,
> Fuel core reviewers would have to also be upstream core reviewers, and
> Fuel CI would have to be allowed to vote on upstream commits. On Fuel
> side, we'd have to complete the reconciliation and modularization of all
> our forked modules, and move all Fuel CI to openstack-infra. The
> potential benefits for both sides are tremendous, but only after we
> essentially merge the two projects. Even if that's achievable, is that
> something whole Puppet OpenStack community is interested in?

* Having people from Fuel team in Puppet OpenStack core team would be
done by meritocracy as any developer in the community actively contributing.
* Having third party CI: we already have TripleO jobs that are run for
every commit in Puppet Modules. Though you'll have to make your CI 100%
public, which is not the case today. Example on
https://review.fuel-infra.org/ with
https://review.fuel-infra.org/#/c/7568/ or any patch, if I want to read
the logs of CI jobs, I can't access to your CI servers:
http://osci-jenkins.srt.mirantis.net:8080/job/7.0.mos-new.build-deb-request/381/
So you would have to expose your servers so the community can debug the
failures.

>> Again, I'm not sure this is the right direction. The official channel
>> for Puppet OpenStack modules is #puppet-openstack and this is the place
>> to be when you're involved in the Puppet OpenStack community.
>> I would suggest to rewrite this thing in "if a bug is discovered in
>> Puppet OpenStack after it's been reported or fixed in Fuel, then folks
>> (from both groups) should collaborate on Puppet OpenStack official
>> channel to fix it upstream".
>> IMHO, Fuel IRC channel should relate to Fuel specific things.
>>
>> As a example, RDO has #rdo-puppet talking about Puppet OpenStack
>> downstream (packstack, etc) things and use #puppet-openstack for
>> upstream related bugs. I think this is the way to go if we want to
>> improve our collaboration.
> 
> Agreed, Fuel developers should use #puppet-openstack to discuss upstream
> work, expecting upstream developers to come to #fuel-dev is not
> reasonable. As discussed above, the trick is to make sure that Fuel
> developers can afford to prioritize the upstream work. Without that,
> collaboration won't happen on any IRC channel.

Well, this is all about it.

>>> If you have specific bugs or commits you'd like to discuss, let's work
>>> them out. I believe I can get the Fuel Library team to agree to do a
>>> walk through all the open bugs in Puppet OpenStack and see if we have
>>> any related fixes or bug reports.
>>
>> Like I already said, I won't share any patch/bug because I don't want to
>> blame anyone here, this is not the goal. Going thru Launchpad and
>> Gerrit, you'll easily see what I mean.
> 
> I think what Matt meant here is that even if we have a policy to
> upstream our changes, people miss things, so it's possible that some of
> our patches were not proposed to upstream, and it would be helpful of
> you to point those cases out so that we can fix that and propose those
> patches properly (referring to the relevant bugs etc.) I see how doing
> that in on a public mailing list can turn into a blame game, what about
> reporting an LP bug against Fuel in the form of "this patch, commit, or
> line of code modifies the code forked from upstream module, please
> report to upstream as per policy"?
> 

I did in my previous replies.

Best regards,
-- 
Emilien Macchi

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150611/7eafc6dd/attachment.pgp>