Open Stack

Hi, all

I'm facing problems with sample config file generation on some projects.

The doc web team is working on finding a way to get all options for all
projects (or as needed) to create more up-to-date and accurate doc config
tables.  They are planning to make the process as generic as possible to
work with the source for any project so as to generate all config options,
so that we can then have the options list converted to docbook format and
update the docs site.

Currently, listing of all register options can be done in Nova and Cinder
using the tools below
https://github.com/openstack/nova/blob/master/tools/conf/generate_sample.sh
https://github.com/openstack/cinder/blob/master/tools/conf/generate_sample.sh

So our plan is to move the "generate_sample.sh" to oslo with an additional
docbook format ouput module which will enable:
./generate_sample.sh /opt/stack/nova
./generate_sample.sh /opt/stack/quantum
./generate_sample.sh /opt/stack/keystone

The module which traverses the target file handed by "generate_sample.sh" is
https://github.com/openstack/oslo-incubator/blob/master/openstack/common/config/generator.py

The "generate_sample.sh" looks for files containing "Opt(" which means that
file has a high chance of defining and registering an option inside. For
instance,

resource_tracker_opts = [
    cfg.IntOpt('reserved_host_disk_mb', default=0,
               help='Amount of disk in MB to reserve for the host'),
    cfg.IntOpt('reserved_host_memory_mb', default=512,
               help='Amount of memory in MB to reserve for the host'),
    cfg.StrOpt('compute_stats_class',
               default='nova.compute.stats.Stats',
               help='Class that will manage stats for the local compute host')
]

CONF = cfg.CONF
CONF.register_opts(resource_tracker_opts)


So if a file is passed to "generator.py", a module object is created by
importing the target file and all attributes in that module object which
are oslo.config.cfg.Opt sub-classes or a list of oslo.config.cfg.Opt
sub-classes are printed out in the sample config file.

Therefore the assumption was that an option will be defined and registered
in the same module.

However, some projects have options defined and registered on different
modules. For instance,
https://github.com/openstack/keystone/blob/master/keystone/common/config.py

Here, the options are defined but instead of directly registering them, it
provides a function which registers the options. And the actual
registration happens in,
https://github.com/openstack/keystone/blob/master/keystone/config.py#L24

Another pattern which is hard to detect the registered options is,
https://github.com/openstack/glance/blob/master/glance/cmd/scrubber.py#L54

This line is not only registering a option inside a function but the option
is also declared inside cfg.register_opt() which is makes it even harder to
detect the option.

How can we solve this problem of not being able to detect options? I have
two suggestions.

First method is to change how generate.py works.
Rake all modules in project which imports oslo.config.cfg and create a
module object of it to inspect the cfg.CONF object inside.
We can cache all the options which were already seen and print out newly
discovered options.
The downside of this solution is that we have to call a private function in
ConfigOpts called _all_opt_infos() to look at all options registered.
https://github.com/openstack/oslo.config/blob/master/oslo/config/cfg.py#L1822
We would have to create a public api version of it.

The second option is to make it a convention that all options should be
declared in global view and also registered in the same module where they
were defined.
However it would be difficult to write a hacking.py routine to check this
and there will be significant amount of code we have to change.

So some questions,
1) Is it worth documenting all the options in a project?
2) Should there be a convention of declaring and registering options? Or
just have generate.py handle things on its own.
3) Any objections of adding a public version of _all_opt_infos()?

-- 
*Intel SSG/STOD/DCST/CIT*
880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
China
+862161166500
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130613/0b0b562b/attachment.html>