Open Stack

On Mon, 2014-02-10 at 10:50 -0500, Doug Hellmann wrote:
>         
>         2) It locates options based on them being defined at the top
>         level of a
>         module (not in function or packed into some object), but it
>         gets info by
>         looking at cfg.CONF, which require registration. This fails if
>         an option
>         is defined by only registered when some function is called.
>         This happens
>         right now with the ip_lib_force_root option in Neutron. Both
>         the config
>         file generator and config docs generator currently crash on
>         Neutron
>         because of this.
> 
> 
> The option definition needs to be in a variable at the module top
> level, even if the registration call happens at run time. Can you file
> a bug against Neutron about that?

The option *is* currently defined at the module top level, though it's
only registered when a function is called. _guess_groups in generator.py
raises an error, because it looks for all instantiated options in the
cfg.CONF object, and it's not there.


>         3) It's completely incapable of finding options that aren't
>         defined or
>         registered until some function is called or some object is
>         instantiated.
>         I don't have a general solution to this, although I do have
>         special-case
>         code that detects projects that use oslo.messaging and ensures
>         those
>         options get registered.
> 
> 
> I did some work to address this under
> https://blueprints.launchpad.net/oslo/+spec/improve-config-discovery-for-docs so that libraries can register entry points that declare configuration options. I think the only library using this feature so far is oslo.messaging, but the other oslo libraries will use it when they graduate from the incubator.

Oh, thanks. That's much cleaner than my workaround for oslo.messaging:

https://review.openstack.org/#/c/68196/



>         To address these issues, I'd like to get oslo.config to know
>         about the
>         defining module for each option. It can get that using
>         something like
>         this in Opt.__init__:
>         
>         for frame in inspect.stack():
>             mod = inspect.getmodule(frame[0])
>             if mod == sys.modules[__name__]:
>                 continue
>             self.module = mod.__name__
>                 break
> 
> 
> I'm not sure we want deployers to have to worry about where the option
> is defined. How would you use the information in the documentation? I
> guess we include it in the sample config output?

I agree that deployers shouldn't have to care about the defining module.
The only reason I've proposed it is that the entire architecture of the
sample config generator is geared around poking into modules looking for
options, because it wants to group options by module. This one addition
to Opt.__init__ would make things simpler and faster for the sample
config generator.

As for the docs, I'm not sure if we want that info in there or not. I
wouldn't push hard for it if it's not already there. Whether we'd bother
with it if it is available is something I'd have to ask the rest of the
docs team.

>         
>         We'd also have to modify __eq__ and __ne__, because it's valid
>         for an
>         option to be defined in two different modules as long as all
>         its
>         parameters are the same. Checking vars() equality would trip
>         up on the
>         module. (There's the further issue that the defining module is
>         non-deterministic in those cases.)
> 
> 
> It's valid for an option to be registered with the same definition
> more than once, but having the definition live in more than one place
> is just asking for maintenance trouble later. Do we actually have this
> situation now?

auth_strategy is defined in both neutron/common/config.py and
neutron/agent/linux/interface.py. It would also be defined in
neutron/agent/metadata/agent.py, except it's defined in a class variable
and only registered by calling main().

>  
>         
>         Then I'd like to get both the config file generator and the
>         config docs
>         generator using something like the OptionsCache class I wrote
>         for the
>         config docs generator:
>         
>         http://git.openstack.org/cgit/openstack/openstack-doc-tools/tree/autogenerate_config_docs/common.py#n88
>         
>         This could be simpler and faster and avoid the crash mentioned
>         above
>         with the module info attached to each option.
> 
> 
> Having 2 programs that discover options in 2 ways is a bad thing, so
> yes, let's combine them in oslo.config and let them share code.

Great. So I think there's three things we'll have to figure out:

1) What's the best way to group things by module? The answer might just
be "It's not really worth grouping things by module."

2) Is it sufficient to just ask cfg.CONF for options, or do we need to
poke around looking for unregistered options? If the latter, where do we
put that poking around code?

3) Where do we put the code that sanitizes the defaults? That's the
other piece of the sample config generator I'm using. It's a _private
function right now, so I'm a bit uneasy about using it. I don't know if
that belongs in oslo.config proper. It's only useful to these generator
programs.

--
Shaun