Open Stack

On 03/04/14 08:48, Doug Hellmann wrote:
> On Wed, Apr 2, 2014 at 9:55 PM, Zane Bitter <zbitter at redhat.com> wrote:
>> We have an issue in Heat where the sample config generator from Oslo is
>> currently broken (see bug #1288586). Unfortunately it turns out that there
>> is no fix to the generator script itself that can do the Right Thing for
>> both Heat and Nova.
>>
>> A brief recap on how the sample config generator works: it goes through all
>> of the files specified and finds all the ConfigOpt objects at the top level.
>> It then searches for them in the registered options, and returns the name of
>> the group in which they are registered. Previously it looked for the
>> identical object being registered, but now it just looks for any equivalent
>> ones. When you register two or more equivalent options, the second and
>> subsequent ones are just ignored by oslo.config.
>>
>> The situation in Heat is that we have a bunch of equivalent options
>> registered in multiple groups. This is because we have a set of options for
>> each client library (i.e. python-novaclient, python-cinderclient, &c.), with
>> each set containing equivalent options (e.g. every client has an
>> "endpoint_type" option for looking up the keystone catalog). This used to
>> work, but now that equivalent options (and not just identical options) match
>> when searching for them in a group, we just end up with multiple copies of
>> each option in the first group to be searched, and none in any of the other
>> groups, in the generated sample config.
>>
>> Nova, on the other hand, has the opposite problem (see bug #1262148). Nova
>> adds the auth middleware from python-keystoneclient to its list of files to
>> search for options. That middleware imports a file from oslo-incubator that
>> registers the option in the default group - a registration that is *not*
>> wanted by the keystone middleware, because it registers an equivalent option
>> in a different group instead (or, as it turns out, as well). Just to make it
>> interesting, Nova uses the same oslo-incubator module and relies on the
>> option being registered in the default group. Of course, oslo-incubator is
>> not a real library, so it gets registered a second time but ignored (since
>> an equivalent one is already present). Crucially, the oslo-incubator file
>> from python-keystoneclient is not on the list of extra modules to search in
>> Nova, so when the generator script was looking for options identical to the
>> ones it found in modules, it didn't see this option at all. Hence the change
>> to looking for equivalent options, which broke Heat.
>>
>> Neither comparing for equivalence nor for identity in the generator script
>> can solve both use cases. It's hard to see what Heat could or should be
>> doing differently. I think it follows that the fix needs to be in either
>> Nova or python-keystoneclient in the first instance.
>>
>> One option I suggested was for the auth middleware to immediately deregister
>> the extra option that had accidentally been registered upon importing a
>> module from oslo-incubator. I put up patches to do this, but it seemed to be
>> generally agreed by Oslo folks that this was a Bad Idea.
>>
>> Another option would be to specifically include the relevant module from
>> keystoneclient.openstack.common when generating the sample config. This
>> seems quite brittle to me.
>>
>> We could fix it by splitting the oslo-incubator module into one that
>> provides the code needed by the auth middleware and one that does the
>> registration of options, but this will likely result in cascading changes to
>> a whole bunch of projects.
>>
>> Does anybody have any thoughts on what the right fix looks like here?
>> Currently, verification of the sample config is disabled in the Heat gate
>> because of this issue, so it would be good to get it resolved.
>>
>> cheers,
>> Zane.
>
> We've seen some similar issues in other projects where the "guessing"
> done by the generator is not matching the newer ways we use
> configuration options. In those cases, I suggested that projects use
> the new entry-point feature that allows them to explicitly list
> options within groups, instead of scanning a set of files. This
> feature was originally added so apps can include the options from
> libraries that use oslo.config (such as oslo.messaging), but it can be
> used for options define by the applications as well.
>
> To define an option discovery entry point, create a function that
> returns a sequence of (group name, option list) pairs. For an example,
> see list_opts() in oslo.messaging [1]. Then define the entry point in
> your setup.cfg under the "oslo.config.opts" namespace [2]. If you need
> more than one function, register them separately.
>
> Then change the way generate_sample.sh is called for your project so
> it passes the -l option [3] once for each name you have given to the
> entry points. So if you have just "heat" you would pass "-l heat" and
> if you have "heat-core" and "heat-some-driver" you would pass "-l
> heat-core -l heat-some-driver".
>
> For application options, you shouldn't mix the -l option with the file
> scanner, since you will end up with duplicate options.
>
> Doug
>
> [1] http://git.openstack.org/cgit/openstack/oslo.messaging/tree/oslo/messaging/opts.py#n56
> [2] http://git.openstack.org/cgit/openstack/oslo.messaging/tree/setup.cfg#n53
> [3] http://git.openstack.org/cgit/openstack/oslo-incubator/tree/tools/config/generate_sample.sh#n25

I gave this a go last week when the issue first came up, but couldn't 
get it working. The patch[1] was a mess because it required an update 
from oslo-incubator first, and the mechanism doesn't seem designed for 
using exclusively - I never figured out how to turn off the file scanner 
and rely purely on the 'library' load points. At that point I realised 
that things weren't actually broken in the gate yet, and I could just 
get things moving again for Icehouse by committing the diff that the 
gate check was asking for, so I did.

Now that we're in Juno the gate is definitely broken, however. And this 
doesn't sound like a long term fix unless everything is moving over to 
it (which I would support) - Heat is actually doing the Right Thing 
already; Nova is unintentionally importing an unwanted option and then 
ignoring it. Don't we need a solution for that?

cheers,
Zane.

[1] http://paste.fedoraproject.org/91147/13964840