[openstack-dev] [chef] Making the Kitchen Great Again: A Retrospective on OpenStack & Chef

Alex Schultz aschultz at redhat.com
Wed Feb 15 16:49:14 UTC 2017

On Wed, Feb 15, 2017 at 9:02 AM, Samuel Cassiba <s at cassiba.com> wrote:
>> On Feb 15, 2017, at 02:07, Thierry Carrez <thierry at openstack.org> wrote:
>> Samuel Cassiba wrote:
>>> [...]
>>> *TL;DR* if you don't want to keep going -
>>> OpenStack-Chef is not in a good place and is not sustainable.
>>> [...]
>> Thanks for sharing, Sam.
> Thanks for taking the time to read and respond. This was as hard to write as it was to read. As time went on, it became apparent that this retrospective needed to exist. It was not written lightly, and does not aim to point fingers.
>> I think that part of the reasons for the situation is that we grew the
>> number of options for deploying OpenStack. We originally only had Puppet
>> and Chef, but now there is Ansible, Juju, and the various
>> Kolla-consuming container-oriented approaches. There is a gravitational
>> attraction effect at play (more users -> more contributors) which
>> currently benefits Puppet, Ansible and Kolla, to the expense of
>> less-popular community-driven efforts like OpenStackChef and
>> OpenStackSalt. I expect this effect to continue. I have mixed feelings
>> about it: on one hand it reduces available technical options, but on the
>> other it allows to focus and raise quality…
> You have a very valid point. One need only look at the trends over the cycles in the User Survey to see this shift in most places. Ansible wins due to sheer simplicity for new deployments, but there are also real business decisions that go behind automation flavors at certain business sizes. This leaves them effectively married to whichever flavor chosen. That shift impacts Puppet’s overall user base, as well, though they had and still have the luxury of maintaining sponsored support at higher numbers.

To chime in on the Puppet side, we've seen a decrease in contributors
over the last several cycles and I have a feeling we'll be in the same
boat in the near future.  The amount of modules that we have to try
and manage versus the amount of folks that we have contributing is
getting to an unmanageable state.  I believe the only way we've gotten
to where we have been is due to the use within Fuel and TripleO.  As
those projects evolve, it directly impacts the ability for the Puppet
modules to remain relevant.  Some could argue that's just the way it
goes and technologies evolve which is true.  But it's also a loss for
many of the newer methods as they are losing all of the historical
knowledge and understanding that went with it and why some patterns
work better than others.  The software wheel, it's getting reinvented
every day.

> Chef’s sponsored support has numbered far fewer. It casts an extremely negative image on OpenStack when someone looks for help at odd hours, or asks something somewhere that none of us have time to track. The answer to that is the point of making noise, to generate conversation about avenues and solutions. I could have kept my fingers aiming at LP, Gerrit and IRC in an attempt to bury my head in the sand. We’re way past the point of denial, perhaps too far, but as long as the results of the User Survey shows Chef, there are still users to support, for now. Operators and deployers will be looking to the source of truth, wherever that is, and right now that source of truth is OpenStack.
>> There is one question I wanted to ask you in terms of community. We
>> maintain in OpenStack a number of efforts that bridge two communities,
>> and where the project could set up its infrastructure / governance in
>> one or the other. In the case of OpenStackChef, you could have set up
>> shop on the Chef community side, rather than on the OpenStack community
>> side. Would you say that living on the OpenStack community side helped
>> you or hurt you ? Did you get enough help / visibility to balance the
>> constraints ? Do you think you would have been more, less or equally
>> successful if you had set up shop more on the Chef community side ?
> We set up under Stackforge, later OpenStack, because the cookbooks evolved alongside OpenStack, as far back as 2011, before my time in the cookbooks. The earliest commits on the now EOL Grizzly branch were quite enlightening, if only Stackalytics had the visuals. Maybe I’m biased, but that’s worth something.
> You’re absolutely correct that we could have pushed more to set up the Chef side of things, and in fact we made several concerted efforts to integrate into the Chef community, up to and including having sponsored contributors, even a PTL. When exploring the Chef side, we found that we faced as much or more friction with the ecosystem, requiring more fundamental changes than we could influence. Chef (the ecosystem) has many great things, but Chef doesn’t OpenStack. Maybe that was the writing on the wall.
> I keep one foot in both Chef and OpenStack, to keep myself as informed as time allows me. It’s clear that even Chef’s long-term cookbook support community is ill equipped to handle OpenStack. The problem? We’re too complex and too far integrated, and none of them know OpenStack. Where does that leave us?

This is a shared concern that I have along with others who have voiced
the concern around features/changes in the OpenStack projects and
their impacts on the deployment tools (and ultimately the end user).
OpenStack continues to get more complex and the support tooling that
has grown around it (ansible, puppet, chef, charms) is struggling to
keep up and maintain because there is not an easy way to stay informed
on all the changes for the entire ecosystem (and there are a ton of
changes).  "Read the release notes" is not a great answer for the end
user.  Neither is "Read the release notes for every project deployed
and figure it out".  I see this problem is a community and culture
problem within OpenStack on and not necessarily a problem with the
folks trying to write the tooling to manage it.


> --
> Best,
> Samuel Cassiba
>> --
>> Thierry Carrez (ttx)
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

More information about the OpenStack-dev mailing list