Open Stack

On 04/22/2013 01:52 PM, Dolph Mathews wrote:
> 
> On Mon, Apr 22, 2013 at 9:11 AM, Mark McLoughlin <markmc at redhat.com
> <mailto:markmc at redhat.com>> wrote:
> 
>     Hi Chmoel,
> 
>     On Mon, 2013-04-22 at 17:35 +0200, Chmouel Boudjnah wrote:
>     > Hi,
>     >
>     > In review :
>     >
>     > https://review.openstack.org/#/c/23986/
>     >
>     > I am trying to bring some update to the ec2 token middleware shipped
>     > with keystone to bring  caching, tests use the oslo.config module
>     > instead of
>     > nova.flags (which is obsolete in grizzly).
>     >
>     > When running the tests I am getting the infamous error :
>     >
>     > 2013-04-18 21:20:55.442 | DuplicateOptError: duplicate option:
>     log-format
>     >
>     >
>     http://logs.openstack.org/23986/3/check/gate-keystone-python26-rhel6/219/console.html.gz
>     >
>     > While reading other bug reports on it I understand why this has been
>     > done, to not get a duplication of config flags, but in my case I just
>     > have a middleware designed for nova run in keystone and somehow
>     > oslo.config parse nova and keystone during the tests config flags when
>     > run by tox.
>     >
>     > And that's the strange thing when I run the indivual test class the
>     > problem does not show up but only when I run the full test suite :
>     >
>     > http://pastie.org/private/x1wdqcrvrzmlk1h6jkazqg
>     >
>     > so I am guessing there is some mix up going on somewhere while runing
>     > the full test suite.
>     >
>     >
>     > The solution/workarounds I can see are :
>     >
>     > 1) Move ec2_token to nova source tree.
>     > 2) do a try: catch(DuplicateError): pass  when parsing the flags.
>     > 3) Don't use nova for wsgi but the one from keystone.
>     > 4) fix tox.
>     >
>     > I don't like 3 because it would assume we need to install keystone
>     > package to use ec2token middleware which should be used only with nova
>     > so then i'm in favor of 1) because 2) is just plain ugly and I have
>     > spent too much time trying to figure out how to do 4) that I am just
>     > avoiding it (infra peps if you have any ideas?).
> 
>     I admit I'm replying here without digging into the details, but the
>     thing that jumps out at me is:
> 
>       keystone/middleware/ec2_token.py:
> 
>         from nova import wsgi
> 
>       tools/test-requires:
> 
>         # ec2token test use nova.wsgi
>         http://tarballs.openstack.org/nova/nova-master.tar.gz#egg=nova
> 
>     We really, really should not have code in keystone which depends on code
>     in nova. Nova is not library code, its interface is not stable.
> 
>     Why can't the ec2 token middleware live in Nova?
> 
> 
> or novaclient or keystoneclient, given the precedence of
> keystoneclient.middleware.auth_token?
>  
> 
> 
>     As for the DuplicateOpt error, there are a couple of ways I can see to
>     avoid it:
> 
>       1) Fix it so that importing the ec2 middleware doesn't import all of
>          keystone's config options via the import in keystone.middleware
> 
> 
> I couldn't find a way to do this when working
> on https://bugs.launchpad.net/nova/+bug/1143998
> 
> I ended up implementing a workaround which might work for ec2_token as
> well? See:
> 
>   https://review.openstack.org/#/c/24701/
>  
> 
> 
>     or
> 
>       2) Make keystone use the common logging infrastructure from Oslo
> 
> 
> +1, but I'd bet that you'd just end up raising a conflict on a different
> config option?

I believe part of the problem is the global config object and testing
different code paths in the same process. I'll dig in next time I'm not
sitting in a car - but I think that it's likely that we can fix this by
properly wrapping the conf object in a fixture like we did in the nova
test suite.

Of course, that'll be some thing we need to do as part of the "migrate
keystone to testtools/fixtures/testr" work ...