Open Stack

----- Original Message -----
> From: "Robert Collins" <robertc at robertcollins.net>
> To: "OpenStack Development Mailing List" <openstack-dev at lists.openstack.org>
> Sent: Monday, April 7, 2014 4:00:30 PM
> Subject: [openstack-dev] [TripleO] config options, defaults, oh my!
> 
> So one interesting thing from the influx of new reviews is lots of
> patches exposing all the various plumbing bits of OpenStack. This is
> good in some ways (yay, we can configure more stuff), but in some ways
> its kindof odd - like - its not clear when
> https://review.openstack.org/#/c/83122/ is needed.
> 
> I'm keen to expose things that are really needed, but i'm not sure
> that /all/ options are needed - what do folk think?

I think we can learn much from some of the more mature configuration management tools in the community on this front. Using puppet as an example here (although I'm sure other tools may do similar things as well)... Take configuration of the Nova API server. There is a direct configuration parameter for 'neutron_metadata_proxy_shared_secret' in the Puppet nova::api class. This parameter is exposed in the class (sort of the equivalent of a TripleO element) directly because it is convenient and many users may want to customize the value. There are however hundreds of Nova config options and most of them aren't exposed as parameters in the various Nova puppet classes. For these it is possible to define a nova_config resource to configure *any* nova.conf parameter in an ad hoc style for your own installation tuning purposes.

I could see us using a similar model in TripleO where our elements support configuring common config elements directly, but we also allow people to tune extra "undocumented" options for their own use. There is always going to be a need for this as people need to tune things for their own installations with options that may not be appropriate for the common set of elements.

Standardizing this mechanism across many of the OpenStack service elements would also make a lot of sense. Today we have this for Nova:

nova:
  verbose: False
    - Print more verbose output (set logging level to INFO instead of default WARNING level).
  debug: False
    - Print debugging output (set logging level to DEBUG instead of default WARNING level).
  baremetal:
    pxe_deploy_timeout: "1200"
  .....

I could see us adding a generic mechanism like this to overlay with the existing (documented) data structure:

nova:
   config:
       default.compute_manager: ironic.nova.compute.manager.ClusterComputeManager
       cells.driver: nova.cells.rpc_driver.CellsRPCDriver

And in this manner a user might be able to add *any* supported config param to the element.


> Also, some things
> really should be higher order operations - like the neutron callback
> to nova right - that should be either set to timeout in nova &
> configured in neutron, *or* set in both sides appropriately, never
> one-half or the other.
> 
> I think we need to sort out our approach here to be systematic quite
> quickly to deal with these reviews.

I totally agree. I was also planning to email the list about this very issue this week :) My email subject was going to be "TripleO templates... an upstream maintenance problem".

For the existing reviews today I think we should be somewhat selective about what parameters we expose as top level within the elements. That said we are missing some rather fundamental features to allow users to configure "undocumented" parameters as well. So we need to solve this problem quickly because there are certainly some configuration corner that users will need.

As is today we are missing some rather fundamental features in os-apply-config and the elements to be able to pull this off. What we really need is a generic INI style template generator. Or perhaps we could use something like augeus or even devstacks simple ini editing functions to pull this off. In any case the idea would be that we allow users to inject their own undocumented config parameters into the various service config files. Or perhaps we could auto-generate mustache templates based off of the upstream sample config files. Many approaches would work here I think...


> 
> Here's an attempt to do so - this could become a developers guide patch.
> 
> Config options in TripleO
> ==================
> 
> Non-API driven configuration falls into four categories:
> A - fixed at buildtime (e.g. ld.so path)
> B - cluster state derived
> C - local machine derived
> D - deployer choices
> 
> For A, it should be entirely done within the elements concerned.
> 
> For B, the heat template should accept parameters to choose the
> desired config (e.g. the Neutron->Nova example able) but then express
> the config in basic primitives in the instance metadata.
> 
> For C, elements should introspect the machine (e.g. memory size to
> determine mysql memory footprint) inside os-refresh-config scripts;
> longer term we should make this an input layer to os-collect-config.
> 
> For D, we need a sensible parameter in the heat template and probably
> direct mapping down to instance metadata.
> 
> But we have a broader question - when should something be configurable at
> all?
> 
> In my mind we have these scenarios:
> 1) There is a single right answer
> 2) There are many right answers
> 
> An example of 1) would be any test-only option like failure injection
> - the production value is always 'off'. For 2), hypervisor driver is a
> great example - anything other than qemu is a valid production value
> :)
> 
> But, it seems to me that these cases actually subdivide further -
> 1a) single right answer, and the default is the right answer
> 1b) single right answer and it is not the default
> 2a) many right answers, and the default is the most/nearly most common one
> 2b) many right answers, and the default is either not one of them or
> is a corner case
> 
> So my proposal here - what I'd like to do as we add all these config
> options to TripleO is to take the care to identify which of A/B/C/D
> they are and code them appropriately, and if the option is one of 1b)
> or 2b) make sure there is a bug in the relevant project about the fact
> that we're having to override a default. If the option is really a
> case of 1a) I'm not sure we want it configurable at all.

Ideally I think anything should be configurable (hard coding things is bad IMO... and it may make TripleO look bad too). That said I think if we follow a plan similar to what I laid out above we'll be able to be restrictive/selective about what we allow into elements as documented config parameters while also supporting the case where users need to potentially customize anything.

All the while I think our experiences in TripleO can be a nice feedback loop into some of the upstream OpenStack projects as well to help give advice on what the defaults should be. We may not always win the "TripleO default should be the upstream default" argument but that is Okay. Perhaps this is where the occasional in-element default (which is overridable) will come in handy.

> 
> Thoughts?
> 
> -Rob
> 
> --
> Robert Collins <rbtcollins at hp.com>
> Distinguished Technologist
> HP Converged Cloud
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>