Open Stack

On Thu, Jun 13, 2013 at 11:53 PM, Anne Gentle <anne at openstack.org> wrote:

>
>
>
> On Thu, Jun 13, 2013 at 12:35 AM, Zhongyue Luo <zhongyue.nah at intel.com>wrote:
>
>> Hi, all
>>
>> I'm facing problems with sample config file generation on some projects.
>>
>> The doc web team is working on finding a way to get all options for all
>> projects (or as needed) to create more up-to-date and accurate doc config
>> tables.  They are planning to make the process as generic as possible to
>> work with the source for any project so as to generate all config options,
>> so that we can then have the options list converted to docbook format and
>> update the docs site.
>>
>> Currently, listing of all register options can be done in Nova and Cinder
>> using the tools below
>>
>> https://github.com/openstack/nova/blob/master/tools/conf/generate_sample.sh
>>
>> https://github.com/openstack/cinder/blob/master/tools/conf/generate_sample.sh
>>
>> So our plan is to move the "generate_sample.sh" to oslo with an
>> additional docbook format ouput module which will enable:
>> ./generate_sample.sh /opt/stack/nova
>> ./generate_sample.sh /opt/stack/quantum
>> ./generate_sample.sh /opt/stack/keystone
>>
>> The module which traverses the target file handed by "generate_sample.sh"
>> is
>>
>> https://github.com/openstack/oslo-incubator/blob/master/openstack/common/config/generator.py
>>
>> The "generate_sample.sh" looks for files containing "Opt(" which means
>> that file has a high chance of defining and registering an option inside.
>> For instance,
>>
>>
>>
>>
>> resource_tracker_opts = [
>>
>>
>>
>>     cfg.IntOpt('reserved_host_disk_mb', default=0,
>>
>>
>>
>>                help='Amount of disk in MB to reserve for the host'),
>>
>>
>>
>>     cfg.IntOpt('reserved_host_memory_mb', default=512,
>>
>>
>>
>>                help='Amount of memory in MB to reserve for the host'),
>>
>>
>>
>>     cfg.StrOpt('compute_stats_class',
>>
>>
>>
>>                default='nova.compute.stats.Stats',
>>
>>
>>
>>                help='Class that will manage stats for the local compute host')
>>
>>
>>
>> ]
>>
>> CONF = cfg.CONF
>>
>>
>>
>> CONF.register_opts(resource_tracker_opts)
>>
>>
>>
>>
>> So if a file is passed to "generator.py", a module object is created by
>> importing the target file and all attributes in that module object which
>> are oslo.config.cfg.Opt sub-classes or a list of oslo.config.cfg.Opt
>> sub-classes are printed out in the sample config file.
>>
>> Therefore the assumption was that an option will be defined and
>> registered in the same module.
>>
>> However, some projects have options defined and registered on different
>> modules. For instance,
>>
>> https://github.com/openstack/keystone/blob/master/keystone/common/config.py
>>
>> Here, the options are defined but instead of directly registering them,
>> it provides a function which registers the options. And the actual
>> registration happens in,
>> https://github.com/openstack/keystone/blob/master/keystone/config.py#L24
>>
>> Another pattern which is hard to detect the registered options is,
>> https://github.com/openstack/glance/blob/master/glance/cmd/scrubber.py#L54
>>
>> This line is not only registering a option inside a function but the
>> option is also declared inside cfg.register_opt() which is makes it even
>> harder to detect the option.
>>
>> How can we solve this problem of not being able to detect options? I have
>> two suggestions.
>>
>> First method is to change how generate.py works.
>> Rake all modules in project which imports oslo.config.cfg and create a
>> module object of it to inspect the cfg.CONF object inside.
>> We can cache all the options which were already seen and print out newly
>> discovered options.
>> The downside of this solution is that we have to call a private function
>> in ConfigOpts called _all_opt_infos() to look at all options registered.
>>
>> https://github.com/openstack/oslo.config/blob/master/oslo/config/cfg.py#L1822
>> We would have to create a public api version of it.
>>
>> The second option is to make it a convention that all options should be
>> declared in global view and also registered in the same module where they
>> were defined.
>> However it would be difficult to write a hacking.py routine to check this
>> and there will be significant amount of code we have to change.
>>
>> So some questions,
>> 1) Is it worth documenting all the options in a project?
>>
>
> Yes, absolutely. Consistency is worthwhile also, though we will certainly
> take what we have, make a configuration guide of it, and iterate.
>
>
>> 2) Should there be a convention of declaring and registering options? Or
>> just have generate.py handle things on its own.
>>
>
> I'd like a convention -- you didn't mention swift or glance in your
> examples. How close or far are these projects? I know both have sample
> configuration files but I don't know how they're made.
>
> Thanks for the efforts here.
>
> Anne
>

I didn't look at swift yet but glance and quantum seems to manage their
sample conf files manually(correct me if I'm wrong). Anyway most of the
options in glance and quantum also follow the option declaration convention
so it won't be that hard to extract options from them I think.


>
>
>>  3) Any objections of adding a public version of _all_opt_infos()?
>>
>> --
>> *Intel SSG/STOD/DCST/CIT*
>> 880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
>> China
>> +862161166500
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
*Intel SSG/STOD/DCST/CIT*
880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
China
+862161166500
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130614/6cd48f8b/attachment.html>