On 28.05.2019 1:35, Emilien Macchi wrote:
(First of all: thanks to Alex Schultz who initiated the discussion during the last PTG).
Context: With the containerization of the services in TripleO, our usage of Puppet is pretty much limited to laying down parameters into configuration files. Our long term goal is to reduce our number of tools in TripleO and converge toward more Ansible and we are currently investigating how to make it happen for Configuration Management, in a backward compatible and non-disruptive way.
Problems: - Our current interface to configure things is very tight to Puppet and Hiera, which would make it complicated to bind it with Hiera. Some of us have tried (to create some Hiera module in Ansible) but clearly this is not the path we want to take as far I know. - We don't use the logic (generally) in the Puppet modules and again only care about the configuration providers (in puppet-openstacklib and for some services, templates files), so the Puppet modules now do too much for what we need in TripleO.
Proposals: - Create an interface in THT that will define what the configuration looks like for each service. The format will be simple and easy to consume from any tool (Puppet to start for backward compatibility but also Ansible and why not etcd one day).
Example with Glance in deployment/glance/glance-api-container-puppet.yaml: param_config: map_merge: - glance_api_config: DEFAULT: enable_v2_api: true log_dir: /var/log/glance oslo_middleware: enable_proxy_headers_parsing: true - glance_image_import_config: image_import_opts: image_import_plugins: {get_param: GlanceImageImportPlugins}
https://review.opendev.org/#/c/660791/29/deployment/glance/glance-api-contai...
It's not tight to Hiera and it's generating param_config which is JSON and consumable by Ansible or etcd later. Here, glance_api_config and glance_image_import_config are the Puppet providers which configure a specific file for Glance but we could imagine a binding for Ansible like: glance_api_config: /etc/glance/glance-api.conf
- Move the services to use this new interface and I think it'll take multiple cycles to do that due to our amount of services.
Things I haven't figured out (yet): - I'm still working on figuring out how we will handle ExtraConfig, service_config_settings and all the Hiera datafiles that we support in puppet/role.role.j2.yaml (ideas welcome). I can only imagine something like: ExtraConfig: - glance_api_config: DEFAULT: log_dir: /var/log/glance-2
Which somehow would override a previous hash. The blocker I found is that map_merge doesn't do deep merges, so if you merge glance_api_config, you overrides all keys... which is problematic if you only want to update one key. I'll investigate that further.
- Backward compatibility for things like "ExtraConfig: glance::api::workers". Since we wouldn't rely on the classes anymore, our users calling the parameters from the classes will be broken. This also needs to be investigated.
- non Puppet OpenStack modules like mysql, apache, etc, mostly use erb templates. This is, for now, out of scope and the plan is to look at OpenStack services first. But ideas welcome here as well. Of course we would prefer to consume upstream roles if they are well tested & maintained.
I believe we should "translate" hiera & puppet configurations we have in t-h-t directly into k8s-native (or at least looking-like) YAML structures, to have those consumable either as k8s config-maps or ansible vars, or etcd keys transparently and interchangeable. That would allow us to omit any additional translation work and intermediate data abstractions in future, when/if we'd decided to convert t-h-t to manage container pods and/or deploy it via k8s operators framework. I know there had been the translation work done for adapting ceph-ansible data structures (right, not really puppet...) into Rook [0], an operator for Kubernetes clusters. So perhaps we should add the ceph clusters deployment automation folks and teams in the loop and leverage their experience with the subject. [0] https://github.com/rook/rook
If you want to review the ongoing work, here are the links: https://review.opendev.org/#/c/660726 https://review.opendev.org/#/c/661377 https://review.opendev.org/#/c/661093 https://review.opendev.org/#/c/660791
Thanks for reading that far, feel free to comment and contribute. I plan to continue this work and evaluate if all of this is actually worth it. -- Emilien Macchi
-- Best regards, Bogdan Dobrelya, Irc #bogdando