Open Stack

> On Feb 15, 2017, at 08:49, Alex Schultz <aschultz at redhat.com> wrote:
> 
> On Wed, Feb 15, 2017 at 9:02 AM, Samuel Cassiba <s at cassiba.com> wrote:
>>> On Feb 15, 2017, at 02:07, Thierry Carrez <thierry at openstack.org> wrote:
>>> 
>>> Samuel Cassiba wrote:
>>>> [...]
>>>> *TL;DR* if you don't want to keep going -
>>>> OpenStack-Chef is not in a good place and is not sustainable.
>>>> [...]
>>> 
>>> Thanks for sharing, Sam.
>> 
>> Thanks for taking the time to read and respond. This was as hard to write as it was to read. As time went on, it became apparent that this retrospective needed to exist. It was not written lightly, and does not aim to point fingers.
>> 
>>> I think that part of the reasons for the situation is that we grew the
>>> number of options for deploying OpenStack. We originally only had Puppet
>>> and Chef, but now there is Ansible, Juju, and the various
>>> Kolla-consuming container-oriented approaches. There is a gravitational
>>> attraction effect at play (more users -> more contributors) which
>>> currently benefits Puppet, Ansible and Kolla, to the expense of
>>> less-popular community-driven efforts like OpenStackChef and
>>> OpenStackSalt. I expect this effect to continue. I have mixed feelings
>>> about it: on one hand it reduces available technical options, but on the
>>> other it allows to focus and raise quality…
>> 
>> You have a very valid point. One need only look at the trends over the cycles in the User Survey to see this shift in most places. Ansible wins due to sheer simplicity for new deployments, but there are also real business decisions that go behind automation flavors at certain business sizes. This leaves them effectively married to whichever flavor chosen. That shift impacts Puppet’s overall user base, as well, though they had and still have the luxury of maintaining sponsored support at higher numbers.
> 
> To chime in on the Puppet side, we've seen a decrease in contributors
> over the last several cycles and I have a feeling we'll be in the same
> boat in the near future.  The amount of modules that we have to try
> and manage versus the amount of folks that we have contributing is
> getting to an unmanageable state.  I believe the only way we've gotten
> to where we have been is due to the use within Fuel and TripleO.  As
> those projects evolve, it directly impacts the ability for the Puppet
> modules to remain relevant.  Some could argue that's just the way it
> goes and technologies evolve which is true.  But it's also a loss for
> many of the newer methods as they are losing all of the historical
> knowledge and understanding that went with it and why some patterns
> work better than others.  The software wheel, it's getting reinvented
> every day.

Thank you for your perspective from the Puppet side. The Survey data alone
paints a certain narrative, and not one I think people want. If OpenStack
deployment choice is down to a popularity contest, the direct result is
fewer avenues back into OpenStack.

Fewer people will think to pick OpenStack as a viable option if it simply
doesn’t support their design, which means less exposure for non-core
projects, less feedback for core projects, rinse, repeat. Developers can and
would coalesce around but a couple of the most popular options, which works
if that’s the way things are intending to go. With that, the OpenStack story
starts to tell less like an ecosystem and more like a distro, bordering on
echo chamber. I don’t think anyone signed up for that. On the other hand,
fewer deployment options allow for more singular focus. Without all that
choice clouding decision-making, one has no way to OpenStack but those few
methods that everyone uses.

>> Chef’s sponsored support has numbered far fewer. It casts an extremely negative image on OpenStack when someone looks for help at odd hours, or asks something somewhere that none of us have time to track. The answer to that is the point of making noise, to generate conversation about avenues and solutions. I could have kept my fingers aiming at LP, Gerrit and IRC in an attempt to bury my head in the sand. We’re way past the point of denial, perhaps too far, but as long as the results of the User Survey shows Chef, there are still users to support, for now. Operators and deployers will be looking to the source of truth, wherever that is, and right now that source of truth is OpenStack.
>> 
>>> There is one question I wanted to ask you in terms of community. We
>>> maintain in OpenStack a number of efforts that bridge two communities,
>>> and where the project could set up its infrastructure / governance in
>>> one or the other. In the case of OpenStackChef, you could have set up
>>> shop on the Chef community side, rather than on the OpenStack community
>>> side. Would you say that living on the OpenStack community side helped
>>> you or hurt you ? Did you get enough help / visibility to balance the
>>> constraints ? Do you think you would have been more, less or equally
>>> successful if you had set up shop more on the Chef community side ?
>> 
>> We set up under Stackforge, later OpenStack, because the cookbooks evolved alongside OpenStack, as far back as 2011, before my time in the cookbooks. The earliest commits on the now EOL Grizzly branch were quite enlightening, if only Stackalytics had the visuals. Maybe I’m biased, but that’s worth something.
>> 
>> You’re absolutely correct that we could have pushed more to set up the Chef side of things, and in fact we made several concerted efforts to integrate into the Chef community, up to and including having sponsored contributors, even a PTL. When exploring the Chef side, we found that we faced as much or more friction with the ecosystem, requiring more fundamental changes than we could influence. Chef (the ecosystem) has many great things, but Chef doesn’t OpenStack. Maybe that was the writing on the wall.
>> 
>> I keep one foot in both Chef and OpenStack, to keep myself as informed as time allows me. It’s clear that even Chef’s long-term cookbook support community is ill equipped to handle OpenStack. The problem? We’re too complex and too far integrated, and none of them know OpenStack. Where does that leave us?
> 
> This is a shared concern that I have along with others who have voiced
> the concern around features/changes in the OpenStack projects and
> their impacts on the deployment tools (and ultimately the end user).
> OpenStack continues to get more complex and the support tooling that
> has grown around it (ansible, puppet, chef, charms) is struggling to
> keep up and maintain because there is not an easy way to stay informed
> on all the changes for the entire ecosystem (and there are a ton of
> changes).  "Read the release notes" is not a great answer for the end
> user.  Neither is "Read the release notes for every project deployed
> and figure it out".  I see this problem is a community and culture
> problem within OpenStack on and not necessarily a problem with the
> folks trying to write the tooling to manage it.

For the cookbooks, every core and non-core project that is supported has to
be tracked. In addition to that, each platform that is supported must be
tracked, for quirks and idiosyncrasies, because they always have them.

Then, there are the cross-project teams that do the packaging, as well as
the teams that do not necessarily ship releases that must be tracked, for
variances in testing methods, mirrors outside the scope of infra, external
dependencies, etc. It can be slightly overwhelming and overloading at times,
even to someone reasonably seasoned. Scale that process, for every ecosystem
in which one desires to exist, by an order of magnitude.

There’s definitely a general undercurrent to all of this, and it’s bigger
than any one person or team to solve. We definitely can’t “read the release
notes” for this.

--
Best,

Samuel Cassiba

> 
> 
> Thanks,
> -Alex
> 
>> --
>> Best,
>> 
>> Samuel Cassiba
>> 
>>> --
>>> Thierry Carrez (ttx)
>>> 
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170215/dc96eec2/attachment.pgp>