Open Stack

On Fri, Nov 17, 2017 at 4:43 AM, Steven Hardy <shardy at redhat.com> wrote:
> On Thu, Nov 16, 2017 at 4:56 PM, James Slagle <james.slagle at gmail.com> wrote:
>> On Thu, Nov 16, 2017 at 8:44 AM, Flavio Percoco <flavio at redhat.com> wrote:
>> What I'm trying to propose is a path towards deprecating the Heat
>> parameter/environment driven and hieradata driven approach to
>> configuring the services. The ansible-role-k8s-* roles should offer a
>> new interface, so I don't think we have to remain tied to Heat
>> forever, so we should consider what we want the long term goal to be
>> in an ideal world, and take some iterative steps to get there.
>
> I agree this is a good time to discuss ways to rationalize the
> toolchain, but I do suspect it may be premature to consider
> derprecating puppet/hiera as AFAIK this doesn't provide any drop-in
> replacement for the config file generation?

I'm not proposing deprecating without a replacement. It's instead
about a way to use the apb roles without having to be locked into
puppet/hieradata/docker-puppet.py. That would be a path towards
deprecation, or we could choose to never deprecate.

I actually don't think the existing roles do lock you into
puppet/hieradata after reviewing:
https://github.com/openstack/ansible-role-k8s-keystone/blob/master/tasks/provision.yml
But, the demos I've seen are all driven by t-h-t and puppet via the
undercloud deploy mechanism. Perhaps an example of using the roles
standalone would be beneficial.

Showing things like:
  - write config files manually and inject them
  - make a manual change directly to a puppet generated (or not)
config file and inject that

Are we planning for an interface and framework that supports these
types of behaviors? Do the existing apb roles offer such an interface?

I think this is something we could consider now so we don't develop a
framework that locks us into a given implementation. If that's already
the case, that's great. I'm trying to get more of a feel of where we
want to go with this work in the long term and how it would integrate
into a more pure Ansible approach. config-download gets us to where we
can treat Heat as ephemeral for the overcloud (if using
deployed-server), where Heat is only used for config and task
generation. A flexible Ansible role architecture can move us further
towards not having to rely on the generated pieces, which are still
pretty complex for a lot of devs and users.

All of the architecture changes we've made over the years in t-h-t
have allowed us to mostly move forward while maintaining some basic
backwards compatibility which is both great and necessary. We need to
continue to do that, but this is also an opportunity to develop
something more flexible that could allow for different tooling
choices.

> I was thinking we'd probably maintain the current docker-puppet.py
> model for this first pass, to reduce the risk of migrating containers
> to k8s, and we could probably refactor things such that this config
> generation via puppet+docker is orchestrated via the ansible roles and
> kubernetes?
>
> The current model is something like:
>
> 1. Run temporary docker container, run puppet, write config files to
> host file system
> 2. Start service container, config files bind mounted into container
> from host filesystem
> 3. Run temporary bootstrapping container (runs puppet, optional step)
>
> (this is simplified for clarity as there are opportunities for some
> other bootstrapping steps)
>
> In the ansible/kubernetes model, it could work like:
>
> 1. Ansible role makes k8s API call creating pod with multiple containers
> 2. Pod starts temporary container that runs puppet, config files
> written out to shared volume
> 3. Service container starts, config consumed from shared volume
> 4. Optionally run temporary bootstrapping container inside pod
>
> This sort of pattern is documented here:
>
> https://kubernetes.io/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume/
>
> The main advantage is we don't have to reimplement config management
> for every single service, but obviously we'd want this to be pluggable
> in the ansible roles so other config management strategies/tools could
> be used instead of our puppet model.

The pattern is fine, and using all the existing tools for config
management is fine. My point is entirely about the last bit. Let's not
force the existing tools as the defined interface for the apb roles.
It's an important first step because once the patches start to get
pushed upstream and landed, we start adopting them as the supported
framework for better or worse. Some of the changes we've had to make
over the years have had to require lots of refactoring in our
frameworks, and others not.

To avoid that, we should consider one aspect of flexiblity now around
the config management tooling and the interface. Let's make sure
that's the case and part of the agreed long term vision. Historically
(for many reasons), we've found that in TripleO, we may consider a
framework flexible but it doesn't really prove to be until we exercise
the interface with a second non-default implementation which ends up
requiring a lot of refactoring work.

What I'm saying is that we should consider those lessons now and ask,
how would someone use these roles outside of Heat or use them without
puppet/hieradata/docker-puppet.py, and does that actually work and is
supported in the framework and interfaces that we're proposing?

-- 
-- James Slagle
--