Open Stack

On 02/16/2017 08:25 AM, Dean Troyer wrote:
> On Thu, Feb 16, 2017 at 7:54 AM, Ian Cordasco <sigmavirus24 at gmail.com> wrote:
>> It seems like a lot of people view "developing OpenStack" as more
>> attractive than "making it easy to deploy OpenStack" (via the
>> deployment tools in the tent). Doing both is really something project
>> teams should do, but I don't think shoving all of the deployment
>> tooling in repo makes that any better frankly. If we put our tooling
>> in repo, we'll likely then start collecting RPM/Debian/etc. packaging
>> in repo as well.
> 
> Part of it _is_ the "deployment isn't shiny/new/features", but there
> is also the historical view lingering that we ship tarballs (and
> realistically, git repo tags) so as not to pick favorites and to keep
> deployment and packaging out of the project repos directly.
> 
> We have >5 different tools that are "mainstream" to consider, not even
> the large project teams have enough expertise to maintain those.  I
> had a hard enough time myself keeping both rpm and apt up to date in
> previous projects, much less recipies, playbooks and the odd shell
> script.

/me puts on old man pants and settles down by the fire to tell an old yarn

Back in the day - and by that, I mean before devstack existed and before
we were using git and before global-requirements - we did debian
packaging (targetting ubuntu) for all (4) of our projects. This is how
the gate worked. If you needed to express that nova had a new
dependency, you had to first go land a change in the packaging repo that
added the dependency (or bumped the version) The gate installed
dependencies on build hosts via "apt-get build-dep $project"

It worked ... to a degree. But then around the Diablo cycle, our friends
at Red Hat showed up. I remember quite clearly a discussion I had with
markmc at what I remember to be the Essex summit in Boston (but it's
possible I'm placing the memory in the wrong room - sorry, I'm getting
old) where he made the very reasonable argument "I've been using RPM for
ages and have never in my life made a debian package - why should I need
to go learn debian packaging to be able to do python development?"

That argument or some form of it has persisted in our DNA ever since. It
is one of the two reasons we switched from interacting with distro
packages to using virtualenvs and pip (the other is what is now a rather
funny story involving the old bzr vs. git argument that I'll happily
tell anyone over alcohol in person)

It's at the root of why TripleO originally came in to existence - and
also why it originally did not use puppet or chef (ansible and salt
weren't reasonable options yet at the time) It's related to the reason
we use devstack in the gate and not puppet or chef or ansible or salt or
juju. We knew that an OpenStack deployment project that used Puppet
would alienate the folks who used Chef at work, and vice-versa.

Quick digression - the real reason we use devstack in the gate is that
it's literally the first thing that the infra team was handed that
actually WORKED. Before devstack, on different occasions we were handed
repos of cookbooks or puppet modules - but none of them came with any
instructions of how to use them. Then devstack arrived and was as simple
as "run stack.sh" - and it actually worked the first time we tried it -
thus the devstack-gate was ultimately born.

As Dean points out - far from having gotten better, this has actually
multiplied. For from not alienating either the Puppet or the Chef
crowed, we REALLY pissed off both crowds when they thought that the
existence of TripleO as an official project disadvantaged people using
other deployment frameworks. (we totally solved the 2 competing standard
problem by adding a third standard) We now also have kolla and fuel and
TripleO in addition to puppet, chef, ansible, salt and juju. Luckily
there does seem to be collaboration on docker images via kolla - and
TripleO's and fuel's use of puppet is in collaboration with
openstack-puppet - so although we're not coalescing on one approach,
we're finding ways to collaborate on the pieces where we can.

We also have not just Ubuntu and Red Hat - but have gained (and recently
lost the maintainer of) Debian, and we have folks from Gentoo and SuSE
deeply involved. When you multiply
kolla+fuel+TripleO+ansible+salt+puppet+chef+juju by
Ubuntu+CentOS+RHEL+Fedora+SLES+OpenSUSE+Gentoo+Debian ... not to mention
the different reasonable versions of each of those ... it quickly
becomes unworkable an unworkable burden for each project developer to
know how to navigate all of them - as much as I wish that were not true.

I'll shut up now. I'm old - it's time for a nap.

> It is also hard to come in to a community and demand that other
> projects suddenly have to support your project just because you showed
> up.  (This works both ways, when we add a "Big Tent" project, now all
> of the downstreams are expected to suddenly add it.)
> 
> The solution I see is to have a mechanism by which important things
> can be communicated from the project developers that can be consumed
> by the packager/deployer community.  Today, however sub-optimal it
> seems to be, that is release notes.  Maybe adding a specific section
> for packaging would be helpful, but in practice that adds one more
> thing to remember to write to a list that is already hard to get devs
> to do.
> 
> Ad Doug mentions elsewhere in this thread, the liaison approach has
> worked in other areas, it may be a useful approach here.  But I know
> as the PTL of a very small project I do not want 5 more hats to wear.
> Maybe one.
> 
> We have encouraged the packaging groups to work together in the past,
> with mixed results, but it seems to me that a lot of the things that
> need to be discovered and learned in new releases will be similar for
> all of the downstream consumers.  Maybe if that downstream community
> could reach a critical mass and suggest a single common way to
> communicate the deployment considerations in a release it would get
> more traction.  And a single project liaison for the collective rather
> than one for each deployment project.
> 
>> I think what would help improve willing bidirectional communication
>> between service and deployment/packaging teams would be an
>> understanding that without the deployment/packaging teams, the service
>> teams will likely be out of a job. People will/can not deploy
>> OpenStack without the tooling these other teams provide.
> 
> True in a literal sense.  This is also true for our users, without
> them nobody will deploy a cloud in the first place.  That has not
> changed anything unfortunately, we still regularly give our users
> miserable experiences because it is too much effort to participate
> with the (now comatose) UX team or prioritize making a common log
> format or reduce the number of knobs available to twiddle to make each
> cloud a unique snowflake.
> 
> We can not sustain being all things to all people.  Given the
> contraction of investment being considered and implemented by many of
> our foundation member companies, we will have to make some hard
> decisions about where to spend the resources we have left.  Some of
> those decision are made for us by those member companies promoting
> their own priorities (your example of Ansible contributions is one).
> But as a community we have an opportunity to express our desires and
> potentially influence those decisions.
> </soapbox>
> 
> dt
>