Open Stack

Emilien,

Thanks for re-raising this. This is a sore subject on the Fuel's side, and
I will sulk in my own shame for not being better here and continue to
encourage my colleagues to be better in this regard.

On Wed, Jun 10, 2015 at 10:45 PM Emilien Macchi <emilien at redhat.com> wrote:

> Hi,
>
> Before reading this e-mail, please keep in mind:
>
> * I have a lot of admiration for Fuel and since I'm working on OpenStack
> Installers (at eNovance and now Red Hat), Fuel is something I always
> consider a good product.
> * This e-mail is about Fuel and Puppet, nothing about Mirantis.
> * I'm writing on behalf of my thoughts, and not on our group.
> * I'm using open mailing-list for open discussion. There is not bad
> spirit in this e-mail and I want to have a productive thread.
>
> I have some concerns I would like to share with you and hopefully find
> some solutions together.
>
> Since I've been working on Puppet OpenStack (2 years now), I see some
> situations that happen - according to me - too often:
>
> * A bug is reported in both Fuel Library and the Puppet module having
> trouble. A patch is provided in Fuel Library (your fork of Puppet
> OpenStack modules) but not in Puppet upstream module. That means you fix
> the bug for Fuel, and not for Puppet OpenStack community. It does not
> happen all the time but quite often.
>

We had started doing this for puppet-openstack and openstack upstreams,
there where some projects that pushed back on this so the team was asked to
stop cross adding projects. Lets identify if this is desired by the puppet
project or otherwise work out a solution. Obviously adding another project
to the same bug is the least work for some of us but it may end up with
spam from improperly handled bugs, there are easily 20-50 bug modifications
a day from the fuel guys so this could become a fire hose if we are not
careful.


> * A patch is submitted in a Puppet module and quite often does not land
> because there is no activity, no tests or is abandonned later because
> fixed in Fuel Library. I've noticed the patch is fixed in Fuel Library
> though.
>

I think this, at least in part, due to the nature in the differences in our
CI systems, since we don't yet rely on rspec the patches that can land in
fuel-library could lack these and the for encourage some one to only
propose to fuel. This is wrong and we need to cover the gap here so the
patch has an equal cost to propose so we can eventually show that
maintenance is lower in upstream.


>
> * RAW copy/paste between upstream modules code and your forks. In term
> of Licensing, I'm even not sure you have the right to do that (I'm not a
> CLA expert though) but well... in term of authorship and statistics on
> code, I'm not sure it's fair. Using submodules with custom patches would
> have been great to respect the authors who created the original code and
> you could have personalize the manifests.
>
(Is possible for us to maintain this data through gerrit with out creating
more than one patch set?)

I don't think anyone on the fuel teams prefers to do this. While it's
unfair, its a technical issue that we are more than open to a better
solution for. Also, if you want to feel better about it (maybe) it costs us
a lot to maintain the forks as it is (Technically and Socially as you
noted).

When we started fuel, we had initially stared with submodules, and they
where a burden to maintain with our tiny team at the time when we had only
50 modules. Now there are 70 in the directory. Some are ours, some our
others. With the numerous upstreams, base commits and effective requirement
to move fast some times the forks won. Now that we are larger and have more
support (people and tools) It would be a great time to step back and
evaluate a better solution.

For the time being, we have the perceived requirement that we are able
patch something quickly if the business need arises. So a solution would
require this functionality at least until we have the trust and support to
do so directly upstream if necessary.


> Note: you can see that I don't give any example because I'm not here to
> blame people or judge anyone.
>
> So the goal of my e-mail is to open the discussion and have a *real*
> collaboration between Fuel team which seems to have a lot of good Puppet
> engineers and Puppet OpenStack team.
>
> We had this kind of discussion at the Summit (in Vancouver and also
> Paris, and even before). Now I would like to officialy know if you are
> interested or not to be more involved.
> I'm also open at any feedback about Puppet OpenStack group and if
> something blocks you to contribute more.
>

We are very much interested, and have been making progress (even if you
cant see it) towards becoming a better member in the community here.

* We have opened externally visible CI for nearly all component testing and
build processes. Once we get run ci from upstream trunk working you will be
able to see everything

* There has been work to pull us up to recent revisions of the modules,
this is necessary for us to be close enough to even propose relevant
patches to upstream with out wasting time to find that it was already
fixed, and to eventually switch to a more friendly system.

* We just got rid of monoithic roles in or composition layer which has
prevented our ability to do any module level CI

* There is an open task to deploy Openstack from trunk/master which means
that we would also need to deploy it using similar puppet manifests

* Similar to the prior, there is an open task to run fuel-library CI for
every upstream module commit

(Yes, I know we have been talking about these two since ATL, we are making
progress, the first two where barriers for this)

* There is another round of manifest syncs in progress which I expect to
yield another rush of patches from us. (more about this shortly)


> We have the same goals, having Puppet modules better. I think it can be
> win/win: you have less diff with upstream and we have more hands in our
> module maintenance.
> Thank you for reading so far, and I'm looking forward to reading from you.
>
> Best regards,
> --
> Emilien Macchi
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-- 

--

Andrew Woodward

Mirantis

Fuel Community Ambassador

Ceph Community
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150616/6e36f336/attachment.html>