Open Stack

(First of all: thanks to Alex Schultz who initiated the discussion during
the last PTG).

Context:
With the containerization of the services in TripleO, our usage of Puppet
is pretty much limited to laying down parameters into configuration files.
Our long term goal is to reduce our number of tools in TripleO and converge
toward more Ansible and we are currently investigating how to make it
happen for Configuration Management, in a backward compatible and
non-disruptive way.

Problems:
- Our current interface to configure things is very tight to Puppet and
Hiera, which would make it complicated to bind it with Hiera. Some of us
have tried (to create some Hiera module in Ansible) but clearly this is not
the path we want to take as far I know.
- We don't use the logic (generally) in the Puppet modules and again only
care about the configuration providers (in puppet-openstacklib and for some
services, templates files), so the Puppet modules now do too much for what
we need in TripleO.

Proposals:
- Create an interface in THT that will define what the configuration looks
like for each service. The format will be simple and easy to consume from
any tool (Puppet to start for backward compatibility but also Ansible and
why not etcd one day).

Example with Glance in deployment/glance/glance-api-container-puppet.yaml:
        param_config:
          map_merge:
            - glance_api_config:
                DEFAULT:
                    enable_v2_api: true
                    log_dir: /var/log/glance
                oslo_middleware:
                  enable_proxy_headers_parsing: true
            - glance_image_import_config:
                image_import_opts:
                  image_import_plugins: {get_param:
GlanceImageImportPlugins}

https://review.opendev.org/#/c/660791/29/deployment/glance/glance-api-container-puppet.yaml

It's not tight to Hiera and it's generating param_config which is JSON and
consumable by Ansible or etcd later.
Here, glance_api_config and glance_image_import_config are the Puppet
providers which configure a specific file for Glance but we could imagine a
binding for Ansible like:
  glance_api_config: /etc/glance/glance-api.conf

- Move the services to use this new interface and I think it'll take
multiple cycles to do that due to our amount of services.

Things I haven't figured out (yet):
- I'm still working on figuring out how we will handle ExtraConfig,
service_config_settings and all the Hiera datafiles that we support in
puppet/role.role.j2.yaml (ideas welcome).
I can only imagine something like:
          ExtraConfig:
            - glance_api_config:
                DEFAULT:
                    log_dir: /var/log/glance-2

Which somehow would override a previous hash. The blocker I found is that
map_merge doesn't do deep merges, so if you merge glance_api_config, you
overrides all keys... which is problematic if you only want to update one
key. I'll investigate that further.

- Backward compatibility for things like "ExtraConfig:
glance::api::workers". Since we wouldn't rely on the classes anymore, our
users calling the parameters from the classes will be broken. This also
needs to be investigated.

- non Puppet OpenStack modules like mysql, apache, etc, mostly use erb
templates. This is, for now, out of scope and the plan is to look at
OpenStack services first. But ideas welcome here as well. Of course we
would prefer to consume upstream roles if they are well tested & maintained.

If you want to review the ongoing work, here are the links:
https://review.opendev.org/#/c/660726
https://review.opendev.org/#/c/661377
https://review.opendev.org/#/c/661093
https://review.opendev.org/#/c/660791

Thanks for reading that far, feel free to comment and contribute. I plan to
continue this work and evaluate if all of this is actually worth it.
-- 
Emilien Macchi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190527/59771c7e/attachment.html>