Open Stack

Excerpts from Doug Hellmann's message of 2017-06-07 14:00:38 -0400:
> Excerpts from Emilien Macchi's message of 2017-06-07 16:42:13 +0200:
> > On Wed, Jun 7, 2017 at 3:31 PM, Doug Hellmann <doug at doughellmann.com> wrote:
> > >
> > > On Jun 7, 2017, at 7:20 AM, Emilien Macchi <emilien at redhat.com> wrote:
> > >
> > > I'm also wondering if we could use oslo-config-generate directly to
> > > generate confd templates, with a new format. So we would have ini,
> > > yaml, json and confd.
> > > "confd" format would be useful when building rpms that we ship in
> > > containers.
> > > "yaml" format would be useful for installers to expose the options
> > > directly to the User Interface, so we know which params OpenStack
> > > provide and we could re-use the data to push it into etcd.
> > >
> > > Would it make sense?
> > >
> > >
> > > I did think about making oslo-config-generator also take the YAML file as
> > > input instead of scanning plugins, and then including all the output formats
> > > in the single command. I haven’t looked to see how much extra complexity
> > > that would add.
> > 
> > Do you mean taking the YAML file that we generate with Ben's work
> > (which would include the parameters values, added by some other
> > tooling maybe)?
> > 
> > I see 2 options at least:
> > 
> > * Let installers to feed etcd with the parameters by using this etcd
> > namespace $CUSTOM_PREFIX + /project/section/parameter (example
> > /node1/keystone/DEFAULT/debug).
> >   And patch oslo.config to be able to generate confd templates with
> > all the options (and ship the template in the package)
> >   I like this option because it provides a way for operators to learn
> > about all possible options in the configuration, with documentation
> > and default values.
> > 
> > * Also let installers to feed etcd but use a standard template like
> > you showed me last week (credits to you for the code):
> > http://paste.openstack.org/show/2KZUQsWYpgrcG2K8TDcE/
> >    I like this option because nothing has to be done in oslo.config,
> > since we use a standard template for all OpenStack configs (see the
> > paste ^)
> > 
> > Thoughts?

[My apologies, I sent this reply directly to Emilien the first time.]

> There are 2 problems with using the generic template.
> 
> 1. In order for confd to work, you have to give it a list of all of the
>    keys in etcd that it should monitor, and that list is
>    application-specific.
> 
> 2. Not all of our configuration values are simple strings or numbers.
>    We have options for managing lists of values, and there is even
>    an Opt class for loading a dictionary for some reason. So,
>    rendering the value in the template will depend on the type of
>    the option.
> 
> Given those constraints, it makes sense to generate a custom template
> for each set of options. We need to generate the confd file anyway, and
> the template can have the correct logic for rendering mult-value
> options.
> 

> One further problem I don't know how to address yet is the applications
> that use dynamic sections in configuration files. I think Cinder
> is still the primary example of this, but other apps may use that
> ability.  I don't know how to tell confd that it needs to look at
> the keys in those groups, since we don't know the names in advance.

The more I think about dealing with configuration files, the more
I think this is going to be the killer issue. If an application
doesn't know what sections go in the file, it can't monitor the
right parts of etcd or any other database looking at individual
settings.

The configmap approach assumes that something publishes the entire
INI file, which at least moves the problem outside of the container
to a place where we've already implemented the logic to deal with
the dynamic aspect of the configuration files.

Using configmap to inject config files, we gain the ability to have
a full accurate INI file but lose the ability to monitor the file
for updates and have mutable options.  Given that we're running the
service in a container, and starting a new container is easy, maybe
that's fine.

Doug