Open Stack

On Wed, Jun 7, 2017 at 3:31 PM, Doug Hellmann <doug at doughellmann.com> wrote:
>
> On Jun 7, 2017, at 7:20 AM, Emilien Macchi <emilien at redhat.com> wrote:
>
> On Tue, Jun 6, 2017 at 6:08 PM, Emilien Macchi <emilien at redhat.com> wrote:
>
> Following-up the session that we had in Boston:
> https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
>
> Here's an update on where we are and what is being done.
>
>
> == Machine Readable Sample Config
>
> Ben's spec has been merged: https://review.openstack.org/#/c/440835/
> And also the code which implements it:
> https://review.openstack.org/#/c/451081/
> He's now working on the documentation on how to use the feature.
>
> $ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml
>
> Here's an example of the output for Keystone config: https://clbin.com/EAfFM
> This feature was asked at the PTG, and it's already done!
>
>
> == Pluggable drivers for oslo.config
>
> Doug's spec has been well written and the feedback from Summit and the
> review was taken in account: https://review.openstack.org/#/c/454897/
> It's now paused because we think we could use confd (with etcd driver)
> to generate configuration files.
>
> Imagine the work done by Ben in Machine Readable Sample Config, that
> would allow us to generate Confd templates for all services (Keystone,
> Nova, etc) via a tool provided in oslo.config with all the options
> available for a namespace.
>
>
> I'm also wondering if we could use oslo-config-generate directly to
> generate confd templates, with a new format. So we would have ini,
> yaml, json and confd.
> "confd" format would be useful when building rpms that we ship in
> containers.
> "yaml" format would be useful for installers to expose the options
> directly to the User Interface, so we know which params OpenStack
> provide and we could re-use the data to push it into etcd.
>
> Would it make sense?
>
>
> I did think about making oslo-config-generator also take the YAML file as
> input instead of scanning plugins, and then including all the output formats
> in the single command. I haven’t looked to see how much extra complexity
> that would add.

Do you mean taking the YAML file that we generate with Ben's work
(which would include the parameters values, added by some other
tooling maybe)?

I see 2 options at least:

* Let installers to feed etcd with the parameters by using this etcd
namespace $CUSTOM_PREFIX + /project/section/parameter (example
/node1/keystone/DEFAULT/debug).
  And patch oslo.config to be able to generate confd templates with
all the options (and ship the template in the package)
  I like this option because it provides a way for operators to learn
about all possible options in the configuration, with documentation
and default values.

* Also let installers to feed etcd but use a standard template like
you showed me last week (credits to you for the code):
http://paste.openstack.org/show/2KZUQsWYpgrcG2K8TDcE/
   I like this option because nothing has to be done in oslo.config,
since we use a standard template for all OpenStack configs (see the
paste ^)

Thoughts?

>
> We could have packaging builds (e.g. RDO distgit) using the tooling
> when building packages so we could ship confd templates in addition of
> ini configuration files.
> When services would start (e.g. in containers), confd would generate
> configuration files from the templates that is part of the container,
> and read the values from etcd.
>
> The benefit of doing this, is that a very little work is required in
> oslo.config to make this happen (only a tool to generate confd
> templates). It could be a first iteration.
> Another benefit is that confd will generate a configuration file when
> the application will start. So if etcd is down *after* the app
> startup, it shouldn't break the service restart if we don't ask confd
> to re-generate the config. It's good for operators who were concerned
> about the fact the infrastructure would rely on etcd. In that case, we
> would only need etcd at the initial deployment (and during lifecycle
> actions like upgrades, etc).
>
> The downside is that in the case of containers, they would still have
> a configuration file within the container, and the whole goal of this
> feature was to externalize configuration data and stop having
> configuration files.
>
>
> == What's next
>
> I see 2 short-term actions that we can work on:
>
> 1) Decide if whether or not confd solution would be acceptable for a
> start. I'm asking Kolla, TripleO, Helm, Ansible projects if they would
> be willing to use this feature. I'm also asking operators to give
> feedback on it.
>
> 2) Investigate how to expose parameters generated by Ben's work on
> Machine Readable Sample Config to the users (without having to
> manually maintain all options) - I think this has to be solved on the
> installers side, but I might be wrong; and also investigate how to
> populate parameters data into etcd. This tool could be provided by
> oslo.config probably.
>
>
>
> Any feedback from folks working on installers or from operators would
> be more than welcome!
>
> Thanks,
> --
> Emilien Macchi
>
>
>
>
> --
> Emilien Macchi
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>



-- 
Emilien Macchi