Open Stack

This looks very promising, thank you. One concern I have is that we
maintain some of the troubleshooting ability that the hieradata files give
us today in the long term.

The hieradata files on overcloud nodes make it convenient to glean facts
about the deployed host. They also act as a canary for when the overcloud
Heat stacks are out of sync with the deployed hosts.

Once we make the param_config consumable by Ansible or etcd, please keep
supportability and troubleshooting in mind. Having local config data in a
convenient place on the deployed host is important.

On Mon, May 27, 2019 at 4:36 PM Emilien Macchi <emilien at redhat.com> wrote:

> (First of all: thanks to Alex Schultz who initiated the discussion during
> the last PTG).
>
> Context:
> With the containerization of the services in TripleO, our usage of Puppet
> is pretty much limited to laying down parameters into configuration files.
> Our long term goal is to reduce our number of tools in TripleO and
> converge toward more Ansible and we are currently investigating how to make
> it happen for Configuration Management, in a backward compatible and
> non-disruptive way.
>
> Problems:
> - Our current interface to configure things is very tight to Puppet and
> Hiera, which would make it complicated to bind it with Hiera. Some of us
> have tried (to create some Hiera module in Ansible) but clearly this is not
> the path we want to take as far I know.
> - We don't use the logic (generally) in the Puppet modules and again only
> care about the configuration providers (in puppet-openstacklib and for some
> services, templates files), so the Puppet modules now do too much for what
> we need in TripleO.
>
> Proposals:
> - Create an interface in THT that will define what the configuration looks
> like for each service. The format will be simple and easy to consume from
> any tool (Puppet to start for backward compatibility but also Ansible and
> why not etcd one day).
>
> Example with Glance in deployment/glance/glance-api-container-puppet.yaml:
>         param_config:
>           map_merge:
>             - glance_api_config:
>                 DEFAULT:
>                     enable_v2_api: true
>                     log_dir: /var/log/glance
>                 oslo_middleware:
>                   enable_proxy_headers_parsing: true
>             - glance_image_import_config:
>                 image_import_opts:
>                   image_import_plugins: {get_param:
> GlanceImageImportPlugins}
>
>
> https://review.opendev.org/#/c/660791/29/deployment/glance/glance-api-container-puppet.yaml
>
> It's not tight to Hiera and it's generating param_config which is JSON and
> consumable by Ansible or etcd later.
> Here, glance_api_config and glance_image_import_config are the Puppet
> providers which configure a specific file for Glance but we could imagine a
> binding for Ansible like:
>   glance_api_config: /etc/glance/glance-api.conf
>
> - Move the services to use this new interface and I think it'll take
> multiple cycles to do that due to our amount of services.
>
> Things I haven't figured out (yet):
> - I'm still working on figuring out how we will handle ExtraConfig,
> service_config_settings and all the Hiera datafiles that we support in
> puppet/role.role.j2.yaml (ideas welcome).
> I can only imagine something like:
>           ExtraConfig:
>             - glance_api_config:
>                 DEFAULT:
>                     log_dir: /var/log/glance-2
>
> Which somehow would override a previous hash. The blocker I found is that
> map_merge doesn't do deep merges, so if you merge glance_api_config, you
> overrides all keys... which is problematic if you only want to update one
> key. I'll investigate that further.
>
> - Backward compatibility for things like "ExtraConfig:
> glance::api::workers". Since we wouldn't rely on the classes anymore, our
> users calling the parameters from the classes will be broken. This also
> needs to be investigated.
>
> - non Puppet OpenStack modules like mysql, apache, etc, mostly use erb
> templates. This is, for now, out of scope and the plan is to look at
> OpenStack services first. But ideas welcome here as well. Of course we
> would prefer to consume upstream roles if they are well tested & maintained.
>
> If you want to review the ongoing work, here are the links:
> https://review.opendev.org/#/c/660726
> https://review.opendev.org/#/c/661377
> https://review.opendev.org/#/c/661093
> https://review.opendev.org/#/c/660791
>
> Thanks for reading that far, feel free to comment and contribute. I plan
> to continue this work and evaluate if all of this is actually worth it.
> --
> Emilien Macchi
>
-- 
Dan Sneddon         |  Senior Principal Software Engineer
dsneddon at redhat.com |  redhat.com/cloud
dsneddon:irc        |  @dxs:twitter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190527/56745b16/attachment-0001.html>