Open Stack

Excerpts from Emilien Macchi's message of 2017-06-07 16:42:13 +0200:
> On Wed, Jun 7, 2017 at 3:31 PM, Doug Hellmann <doug at doughellmann.com> wrote:
>> 
>> On Jun 7, 2017, at 7:20 AM, Emilien Macchi <emilien at redhat.com> wrote:
>> 
>> On Tue, Jun 6, 2017 at 6:08 PM, Emilien Macchi <emilien at redhat.com> wrote:
>> 
>> Following-up the session that we had in Boston:
>> https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
>> 
>> Here's an update on where we are and what is being done.
>> 
>> 
>> == Machine Readable Sample Config
>> 
>> Ben's spec has been merged: https://review.openstack.org/#/c/440835/
>> And also the code which implements it:
>> https://review.openstack.org/#/c/451081/
>> He's now working on the documentation on how to use the feature.
>> 
>> $ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml
>> 
>> Here's an example of the output for Keystone config: https://clbin.com/EAfFM
>> This feature was asked at the PTG, and it's already done!
>> 
>> 
>> == Pluggable drivers for oslo.config
>> 
>> Doug's spec has been well written and the feedback from Summit and the
>> review was taken in account: https://review.openstack.org/#/c/454897/
>> It's now paused because we think we could use confd (with etcd driver)
>> to generate configuration files.
>> 
>> Imagine the work done by Ben in Machine Readable Sample Config, that
>> would allow us to generate Confd templates for all services (Keystone,
>> Nova, etc) via a tool provided in oslo.config with all the options
>> available for a namespace.
>> 
>> 
>> I'm also wondering if we could use oslo-config-generate directly to
>> generate confd templates, with a new format. So we would have ini,
>> yaml, json and confd.
>> "confd" format would be useful when building rpms that we ship in
>> containers.
>> "yaml" format would be useful for installers to expose the options
>> directly to the User Interface, so we know which params OpenStack
>> provide and we could re-use the data to push it into etcd.
>> 
>> Would it make sense?
>> 
>> 
>> I did think about making oslo-config-generator also take the YAML file as
>> input instead of scanning plugins, and then including all the output formats
>> in the single command. I haven’t looked to see how much extra complexity
>> that would add.
> 
> Do you mean taking the YAML file that we generate with Ben's work
> (which would include the parameters values, added by some other
> tooling maybe)?
> 
> I see 2 options at least:
> 
> * Let installers to feed etcd with the parameters by using this etcd
> namespace $CUSTOM_PREFIX + /project/section/parameter (example
> /node1/keystone/DEFAULT/debug).
>  And patch oslo.config to be able to generate confd templates with
> all the options (and ship the template in the package)
>  I like this option because it provides a way for operators to learn
> about all possible options in the configuration, with documentation
> and default values.
> 
> * Also let installers to feed etcd but use a standard template like
> you showed me last week (credits to you for the code):
> http://paste.openstack.org/show/2KZUQsWYpgrcG2K8TDcE/
>   I like this option because nothing has to be done in oslo.config,
> since we use a standard template for all OpenStack configs (see the
> paste ^)
> 
> Thoughts?

There are 2 problems with using the generic template.

1. In order for confd to work, you have to give it a list of all of the
  keys in etcd that it should monitor, and that list is
  application-specific.

2. Not all of our configuration values are simple strings or numbers.
  We have options for managing lists of values, and there is even
  an Opt class for loading a dictionary for some reason. So,
  rendering the value in the template will depend on the type of
  the option.

Given those constraints, it makes sense to generate a custom template
for each set of options. We need to generate the confd file anyway, and
the template can have the correct logic for rendering mult-value
options.

One further problem I don't know how to address yet is the applications
that use dynamic sections in configuration files. I think Cinder
is still the primary example of this, but other apps may use that
ability.  I don't know how to tell confd that it needs to look at
the keys in those groups, since we don't know the names in advance.

Doug