Open Stack

On 28.05.2019 1:35, Emilien Macchi wrote:
> (First of all: thanks to Alex Schultz who initiated the discussion 
> during the last PTG).
> 
> Context:
> With the containerization of the services in TripleO, our usage of 
> Puppet is pretty much limited to laying down parameters into 
> configuration files.
> Our long term goal is to reduce our number of tools in TripleO and 
> converge toward more Ansible and we are currently investigating how to 
> make it happen for Configuration Management, in a backward compatible 
> and non-disruptive way.
> 
> Problems:
> - Our current interface to configure things is very tight to Puppet and 
> Hiera, which would make it complicated to bind it with Hiera. Some of us 
> have tried (to create some Hiera module in Ansible) but clearly this is 
> not the path we want to take as far I know.
> - We don't use the logic (generally) in the Puppet modules and again 
> only care about the configuration providers (in puppet-openstacklib and 
> for some services, templates files), so the Puppet modules now do too 
> much for what we need in TripleO.
> 
> Proposals:
> - Create an interface in THT that will define what the configuration 
> looks like for each service. The format will be simple and easy to 
> consume from any tool (Puppet to start for backward compatibility but 
> also Ansible and why not etcd one day).
> 
> Example with Glance in deployment/glance/glance-api-container-puppet.yaml:
>          param_config:
>            map_merge:
>              - glance_api_config:
>                  DEFAULT:
>                      enable_v2_api: true
>                      log_dir: /var/log/glance
>                  oslo_middleware:
>                    enable_proxy_headers_parsing: true
>              - glance_image_import_config:
>                  image_import_opts:
>                    image_import_plugins: {get_param: 
> GlanceImageImportPlugins}
> 
> https://review.opendev.org/#/c/660791/29/deployment/glance/glance-api-container-puppet.yaml
> 
> It's not tight to Hiera and it's generating param_config which is JSON 
> and consumable by Ansible or etcd later.
> Here, glance_api_config and glance_image_import_config are the Puppet 
> providers which configure a specific file for Glance but we could 
> imagine a binding for Ansible like:
>    glance_api_config: /etc/glance/glance-api.conf
> 
> - Move the services to use this new interface and I think it'll take 
> multiple cycles to do that due to our amount of services.
> 
> Things I haven't figured out (yet):
> - I'm still working on figuring out how we will handle ExtraConfig, 
> service_config_settings and all the Hiera datafiles that we support in 
> puppet/role.role.j2.yaml (ideas welcome).
> I can only imagine something like:
>            ExtraConfig:
>              - glance_api_config:
>                  DEFAULT:
>                      log_dir: /var/log/glance-2
> 
> Which somehow would override a previous hash. The blocker I found is 
> that map_merge doesn't do deep merges, so if you merge 
> glance_api_config, you overrides all keys... which is problematic if you 
> only want to update one key. I'll investigate that further.
> 
> - Backward compatibility for things like "ExtraConfig: 
> glance::api::workers". Since we wouldn't rely on the classes anymore, 
> our users calling the parameters from the classes will be broken. This 
> also needs to be investigated.
> 
> - non Puppet OpenStack modules like mysql, apache, etc, mostly use erb 
> templates. This is, for now, out of scope and the plan is to look at 
> OpenStack services first. But ideas welcome here as well. Of course we 
> would prefer to consume upstream roles if they are well tested & maintained.

I believe we should "translate" hiera & puppet configurations we have in 
t-h-t directly into k8s-native (or at least looking-like) YAML 
structures, to have those consumable either as k8s config-maps or 
ansible vars, or etcd keys transparently and interchangeable. That would 
allow us to omit any additional translation work and intermediate data 
abstractions in future, when/if we'd decided to convert t-h-t to manage 
container pods and/or deploy it via k8s operators framework.

I know there had been the translation work done for adapting 
ceph-ansible data structures (right, not really puppet...) into Rook 
[0], an operator for Kubernetes clusters. So perhaps we should add the 
ceph clusters deployment automation folks and teams in the loop and 
leverage their experience with the subject.

[0] https://github.com/rook/rook

> 
> If you want to review the ongoing work, here are the links:
> https://review.opendev.org/#/c/660726
> https://review.opendev.org/#/c/661377
> https://review.opendev.org/#/c/661093
> https://review.opendev.org/#/c/660791
> 
> Thanks for reading that far, feel free to comment and contribute. I plan 
> to continue this work and evaluate if all of this is actually worth it.
> -- 
> Emilien Macchi


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando