<div dir="ltr"><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Feb 7, 2014 at 11:35 AM, Shaun McCance <span dir="ltr"><<a href="mailto:shaunm@gnome.org" target="_blank">shaunm@gnome.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">I've been working on the code that generates our configuration docs. A<br>
couple week ago, I rewrote most of it to address some major problems,<br>
most notably its inability to distinguish same-named options in<br>
different groups.<br>
<br>
I've tried to reuse bits from the oslo-incubator code to generate sample<br>
config files, but it's been tricky. Failing that, I've tried to write<br>
the docs generator in a way where its code could be moved somewhere and<br>
reused by multiple projects.<br></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">oslo.config seems like a good home for that. I want to move the sample config generator there, too, so if we can share code between them and have 2 programs that generate different output formats that would probably be best.</div>
</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
The way the config file generator in oslo-incubator works right now has<br>
some major problems, which I'd like to address:<br>
<br>
1) Because of the way it iterates over modules, then over all options,<br>
it's considerably slower than it should be. The config file generator<br>
groups by defining module, so it needs that info. The config docs<br>
gnerator doesn't do anything with the defining module right now, but I<br>
wouldn't rule out the possibility in the future. </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
2) It locates options based on them being defined at the top level of a<br>
module (not in function or packed into some object), but it gets info by<br>
looking at cfg.CONF, which require registration. This fails if an option<br>
is defined by only registered when some function is called. This happens<br>
right now with the ip_lib_force_root option in Neutron. Both the config<br>
file generator and config docs generator currently crash on Neutron<br>
because of this.<br></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">The option definition needs to be in a variable at the module top level, even if the registration call happens at run time. Can you file a bug against Neutron about that?</div>
<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
3) It's completely incapable of finding options that aren't defined or<br>
registered until some function is called or some object is instantiated.<br>
I don't have a general solution to this, although I do have special-case<br>
code that detects projects that use oslo.messaging and ensures those<br>
options get registered.<br></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">I did some work to address this under <a href="https://blueprints.launchpad.net/oslo/+spec/improve-config-discovery-for-docs">https://blueprints.launchpad.net/oslo/+spec/improve-config-discovery-for-docs</a> so that libraries can register entry points that declare configuration options. I think the only library using this feature so far is oslo.messaging, but the other oslo libraries will use it when they graduate from the incubator.</div>
</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
To address these issues, I'd like to get oslo.config to know about the<br>
defining module for each option. It can get that using something like<br>
this in Opt.__init__:<br>
<br>
for frame in inspect.stack():<br>
mod = inspect.getmodule(frame[0])<br>
if mod == sys.modules[__name__]:<br>
continue<br>
self.module = mod.__name__<br>
break<br></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">I'm not sure we want deployers to have to worry about where the option is defined. How would you use the information in the documentation? I guess we include it in the sample config output?</div>
</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
We'd also have to modify __eq__ and __ne__, because it's valid for an<br>
option to be defined in two different modules as long as all its<br>
parameters are the same. Checking vars() equality would trip up on the<br>
module. (There's the further issue that the defining module is<br>
non-deterministic in those cases.)<br></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">It's valid for an option to be registered with the same definition more than once, but having the definition live in more than one place is just asking for maintenance trouble later. Do we actually have this situation now?</div>
</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Then I'd like to get both the config file generator and the config docs<br>
generator using something like the OptionsCache class I wrote for the<br>
config docs generator:<br>
<br>
<a href="http://git.openstack.org/cgit/openstack/openstack-doc-tools/tree/autogenerate_config_docs/common.py#n88" target="_blank">http://git.openstack.org/cgit/openstack/openstack-doc-tools/tree/autogenerate_config_docs/common.py#n88</a><br>
<br>
This could be simpler and faster and avoid the crash mentioned above<br>
with the module info attached to each option.<br></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">Having 2 programs that discover options in 2 ways is a bad thing, so yes, let's combine them in oslo.config and let them share code.</div>
<div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Doug</div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Is any part of this feasible? Has anybody else worked on resolving these<br>
types of issues with the config file generator?<br>
<br>
--<br>
Shaun<br>
<br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote></div><br></div></div>