Open Stack

On Fri, Feb 7, 2014 at 11:35 AM, Shaun McCance <shaunm at gnome.org> wrote:

> I've been working on the code that generates our configuration docs. A
> couple week ago, I rewrote most of it to address some major problems,
> most notably its inability to distinguish same-named options in
> different groups.
>
> I've tried to reuse bits from the oslo-incubator code to generate sample
> config files, but it's been tricky. Failing that, I've tried to write
> the docs generator in a way where its code could be moved somewhere and
> reused by multiple projects.
>

oslo.config seems like a good home for that. I want to move the sample
config generator there, too, so if we can share code between them and have
2 programs that generate different output formats that would probably be
best.


>
> The way the config file generator in oslo-incubator works right now has
> some major problems, which I'd like to address:
>
> 1) Because of the way it iterates over modules, then over all options,
> it's considerably slower than it should be. The config file generator
> groups by defining module, so it needs that info. The config docs
> gnerator doesn't do anything with the defining module right now, but I
> wouldn't rule out the possibility in the future.


> 2) It locates options based on them being defined at the top level of a
> module (not in function or packed into some object), but it gets info by
> looking at cfg.CONF, which require registration. This fails if an option
> is defined by only registered when some function is called. This happens
> right now with the ip_lib_force_root option in Neutron. Both the config
> file generator and config docs generator currently crash on Neutron
> because of this.
>

The option definition needs to be in a variable at the module top level,
even if the registration call happens at run time. Can you file a bug
against Neutron about that?



>
> 3) It's completely incapable of finding options that aren't defined or
> registered until some function is called or some object is instantiated.
> I don't have a general solution to this, although I do have special-case
> code that detects projects that use oslo.messaging and ensures those
> options get registered.
>

I did some work to address this under
https://blueprints.launchpad.net/oslo/+spec/improve-config-discovery-for-docsso
that libraries can register entry points that declare configuration
options. I think the only library using this feature so far is
oslo.messaging, but the other oslo libraries will use it when they graduate
from the incubator.


>
> To address these issues, I'd like to get oslo.config to know about the
> defining module for each option. It can get that using something like
> this in Opt.__init__:
>
> for frame in inspect.stack():
>     mod = inspect.getmodule(frame[0])
>     if mod == sys.modules[__name__]:
>         continue
>     self.module = mod.__name__
>         break
>

I'm not sure we want deployers to have to worry about where the option is
defined. How would you use the information in the documentation? I guess we
include it in the sample config output?


>
> We'd also have to modify __eq__ and __ne__, because it's valid for an
> option to be defined in two different modules as long as all its
> parameters are the same. Checking vars() equality would trip up on the
> module. (There's the further issue that the defining module is
> non-deterministic in those cases.)
>

It's valid for an option to be registered with the same definition more
than once, but having the definition live in more than one place is just
asking for maintenance trouble later. Do we actually have this situation
now?


>
> Then I'd like to get both the config file generator and the config docs
> generator using something like the OptionsCache class I wrote for the
> config docs generator:
>
>
> http://git.openstack.org/cgit/openstack/openstack-doc-tools/tree/autogenerate_config_docs/common.py#n88
>
> This could be simpler and faster and avoid the crash mentioned above
> with the module info attached to each option.
>

Having 2 programs that discover options in 2 ways is a bad thing, so yes,
let's combine them in oslo.config and let them share code.

Doug



>
> Is any part of this feasible? Has anybody else worked on resolving these
> types of issues with the config file generator?
>
> --
> Shaun
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140210/481a9e02/attachment.html>