Open Stack

Dmitry, thank you for taking your time. Please read inline:

On 06/11/2015 07:12 PM, Dmitry Borodaenko wrote:
> First of all, thank you Emilien for bringing this up, and thank you Matt for
> confirming our commitment to collaborate with puppet-openstack and other
> Puppet modules that Fuel developers consider upstream.
> 
> I'd like to add some more concrete examples of what Fuel team has
> already done, is doing, and what more we can do to improve our
> collaboration with other Puppet developers.
> 
> As Matt has pointed out, sharing bugfixes for forked Puppet modules is
> already a requirement for all Fuel developers:
> 
> https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Fuel_library_for_puppet_manifests

The documentation is good, and well written.

> Here are the key bits that are specifically designed to simplify
> reconciling of changes between Fuel and its upstream:
> 
> 1) "If you are adding a module that is the work of another project and
> is already tracked in separate repo (...) review should also contain the
> commit hash from the upstream repo in the commit message". Using this
> reference to upstream commit, you can cleanly and automatically isolate
> Fuel specific changes to that module into a patch series with two useful properties:
> 
>    a) You can associate each deviant line with a commit in fuel-library
>    repository that would refer to specific LP bug or blueprint and
>    otherwise explain the change.
> 
>    b) The whole patch series can be cleanly applied on top of the
>    specified commit in upstream git. This means you can automatically
>    graft a branch made out of the patch series onto upstream git, and
>    then use git rebase to make that branch mergeable with the current
>    upstream git head.

I spent some time to read Fuel Library, specially its custom manifests
[1] and I don't see any example of these commits. I'm pretty sure I've
missed them, can you give some example?

[1]
https://github.com/stackforge/fuel-library/commits/master/deployment/puppet/openstack/manifests/

> 2) "submit patch to upstream module first, and once merged or +2
> recieved from a core reviewer, the patch should be backported to Fuel
> library as well". Aside from the obvious benefit of immediate
> contribution to upstream, this rule helps to keep Fuel specific changes
> confined within Fuel-specific modules, and discourages arbitrary
> deviations of forked modules from upstream.

Please let me show you an example of a bug that has been fixed in Fuel,
but not in upstream (puppet-nova here), *after* the new Fuel policy
(mentioned by Matthew, October 2014).

The bug: https://bugs.launchpad.net/fuel/+bug/1378962
"Node Libvirt Unique UUID Not Generated On Deployment"

Patch in Fuel, merged & released: https://review.openstack.org/#/c/128640/
Patch in puppet-nova, with negative feedback, without any +2 and staled
for October 2014: https://review.openstack.org/#/c/131710

This is an example and again, I don't blame anyone here. I respect
people who did the patches and I'm happy it's fixed in Fuel.

> It's important to understand that Fuel's "How to contribute" guide is
> not just a set of recommendations for external contributors: it is the
> primary definition of the engineering process that all Fuel developers,
> within and without Mirantis, are expected to follow. If you come across
> a violation of these rules in Fuel, it is a bug, you're more than
> welcome to report it Launchpad, and Fuel developers will address it.
> 
> I understand that even with these rules in place the process isn't
> perfect, even if it were followed flawlessly. If you have ideas on what
> else we can tweak to make upstream's life easier, please share, we're
> always looking for ways to improve our processes.

Like puppet-murano?
Someone from Fuel team created first the module in Fuel, 6 months ago
[1] and 3 months later someone from Fuel team  created an empty
repository in Stackforge [2]. By the way, Puppet OpenStack community
does not have core permissions on this module and it's own by Murano team.

[1]
https://github.com/stackforge/fuel-library/tree/master/deployment/puppet/murano/
[2] https://review.openstack.org/#/c/155688/

Again another example, where I'm very happy to see the module in Fuel
but nothing in Stackforge leveraged by Puppet OpenStack community.

> I hope you understand that there are some tradeoffs that are not worth
> considering. While we can (and do) set aside some time in every release
> to reconcile our local changes with upstream, we can't allow any
> external factors to affect our ability to meet our own deadlines. For
> example, we can and should push all our local changes to upstream, but
> we can't hold back a fix for a Critical bug in Fuel until it is found
> acceptable by upstream. We can further minimize the delta between our
> fork and an upstream module, even eliminate the delta altogether when
> possible, but we have to keep a copy of the module in a repository that
> we control. We can and should make a more efficient use of the time our
> engineers spend interacting with upstream, but only as long as that does
> not subtract from the effort we put into Fuel project's priorities.

I've been working on OpenStack installers for 2 years now [1] [2], I
understand what is a product deadline.

[1] http://spinalstack.enovance.com
[2] https://www.rdoproject.org/RDO-Manager

Let me clarify two things:
* We both agree Fuel started wrong by forking Puppet OpenStack modules
and now Fuel is trying the best to catch-up to upstream. I'm totally
good with that and I'm really happy you changed your methodology.
* I understand Fuel team wants their product awesome (who doesn't?) and
they have deadlines but I'm pretty sure (and this is the goal of this
thread) we can do something together.

Until now, I'm still concerned about 'are we really going to have
collaboration in future' or all of these are words (this e-mail, Fuel
documentation in Wiki) and just words that won't make any change.

> At the same time, it's important for Fuel developers to understand that
> even though Fuel source code is publicly available and tracking and
> minimizing deviation from upstream is encoded in our engineering
> process, it doesn't mean there's no room for improvement. Saying "don't
> worry, be happy, we've got it under control, the ball is on your side"
> is not good enough.

So in that case I would be more than happy to hear from you about a
future plan to really collaborate.

> P.S. There's been more emails on the thread while I was writing this,
> and some points raised are not covered in this email, I'll try to
> address them in another followup.

Indeed, I'm also interested to know what people think about the RAW
copy/paste of Puppet OpenStack modules [1].

[1]
https://github.com/stackforge/fuel-library/commit/e2cbdd06495b749e9439cc2e8f69a94b58ed78fe


Thank you for your reply, I'm happy we can have a discussion.
Best regards,
-- 
Emilien Macchi

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150611/518b6aba/attachment.pgp>