[requirements][oslo] Inclusion of CONFspirator in openstack/requirements
adriant at catalystcloud.nz
Fri Aug 21 04:59:24 UTC 2020
On 21/08/20 9:16 am, Ben Nemec wrote:
> In general, given the complexity of what you're talking about I think
> a driver plugin would be the way to go, as opposed to trying to fit
> this all in with the core oslo.config functionality (assuming you/we
> decide to pursue integrating at all).
> There were a few other things you mentioned as features of the
> library. The following are some off-the-cuff thoughts without having
> looked too closely at the library, so if they're nonsense that's my
> "because I have plugins that need to dynamically load config on
> startup, which then has to be lazy_loaded"
> Something like this could probably be done. I believe this is kind of
> how the castellan driver in oslo.config works. Config values are
> looked up and cached on-demand, as opposed to all at once.
This one is a little weird, but essentially the way this works this in
I load the config so I can start the app and go through base logic and
loading plugins, but the config groups that are pulled from plugins
aren't added to my config group tree until AFTER the config has already
been loaded. So part of the config is usable, but some hasn't yet fully
been loaded until after the plugins are done, and then that subtree will
lazy_load itself when first accessed. It means that until a given
lazy_loaded group is actually accessed as config, the config tree
underneath it can still have config options added. It's likely not too
crazy to do this in oslo, and have groups only read from the cached
source (loaded file dict) when first accessed.
> "I also have weird overlay logic for defaults that can be overridden"
> My knee-jerk reaction to this is that oslo.config already supports
> overriding defaults, so I assume there's something about your use case
> that didn't mesh with that functionality? Or is this part of
> oslo.config that you reused?
Sooo, this one is a little special because what this feature lets you do
is take any group in the config tree once loaded, and call the overlay
function on it with either a dict, or another group. The returned value
will be a deep copy of the config tree with the values present in the
given dict/group overlaid on that original group. As in a depth first
dict update, where only keys that exist on the overriding dict will be
updated in the copy of the original dict.
I need to write the docs for this... with a sane example, but here is my
unit test for it:
I use this in Adjutant by having some config groups which are a default
for something, and another place where many things can override that
default via another group in the config, so I create an overlay copy and
pass that to the place that needs the config it it's most specific case
when the code actually pulls values from the conf group I passed it.
I hope that helps, because it is in my mind a little odd to explain, but
it allows some useful things when reusing actions in different tasks.
> "I also have nested config groups that need to be named dynamically to
> allow plugin classes to be extended without subclasses sharing the
> same config group name."
> Minus the nesting part, this is also something being done with
> oslo.config today. The config driver feature actually uses dynamically
> named groups, and I believe at least Cinder is using them too. They do
> cause a bit of grief for some config tools, like the validator, but it
> is possible to do something like this.
Cool! yeah I implemented that feature in my code because I ran into a
case of confliciting namespaces and needed to find a better way to
handle subclasses that needed to have different dynamic names for their
> Now, as to the question of whether we should try to integrate this
> library with oslo.config: I don't know. Helpful, right?
That's mostly where we got to last time I asked in #openstack-oslo!
> I think answering that question definitively would take a deeper dive
> into what the new library is doing differently than I can commit to.
> As I noted above, I don't think the things you're doing are so far out
> in left field that it would be unreasonable to consider integrating
> into oslo.config, but the devil is in the details and there are a lot
> of details here that I don't know anything about. For example, will
> the oslo.config data model even accommodate nested groups? I suspect
> it doesn't now, but could it? Probably, but I can't say how
> difficult/disruptive it would be.
I looked into it briefly, and to do what I wanted, while also
maintaining oslo.config how it was... ended up a bit messy, so I gave up
because it would take too long and the politics of trying to get it
merged/reviewed wouldn't be worth the effort most likely.
> If someone wanted to make incremental progress toward integration, I
> think writing a basic YAML driver for oslo.config would be a good
> start. It would be generally useful even if this work doesn't go
> anywhere, and it would provide a basis for a hypothetical future
> YAML-based driver providing CONFspirator functionality. From there we
> could start looking to integrate more advanced features one at a time.
> Apologies for the wall of text. I hope you got something out of this
> before your eyes glazed over.
Thanks for the wall of text! It was useful!
I think ultimately it may be safer just maintaining my own separate
library. For people who don't want to use .ini and prefer yaml/toml it's
simpler, and for people who prefer .ini and don't need nesting etc, it's
safer to keep oslo.config as it is. If someone does find anything I've
done that makes sense in oslo.config I'd be happy to work porting it
over, but I don't want to make structural changes to it. I'll always
keep an eye on oslo.config, and may occasionally steal the odd idea if
you add something cool, but I think other than my use of types.py and
some of the Opt classes as a base, my code has diverged quite a bit.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the openstack-discuss