[openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

Sean Dague sean at dague.net
Thu Sep 11 11:48:59 UTC 2014


On 09/10/2014 11:55 AM, Steven Hardy wrote:
> On Wed, Sep 10, 2014 at 10:14:32AM -0400, Sean Dague wrote:
>> Going through the untriaged Nova bugs, and there are a few on a similar
>> pattern:
>>
>> Nova operation in progress.... takes a while
>> Crosses keystone token expiration time
>> Timeout thrown
>> Operation fails
>> Terrible 500 error sent back to user
> 
> We actually have this exact problem in Heat, which I'm currently trying to
> solve:
> 
> https://bugs.launchpad.net/heat/+bug/1306294
> 
> Can you clarify, is the issue either:
> 
> 1. Create novaclient object with username/password
> 2. Do series of operations via the client object which eventually fail
> after $n operations due to token expiry
> 
> or:
> 
> 1. Create novaclient object with username/password
> 2. Some really long operation which means token expires in the course of
> the service handling the request, blowing up and 500-ing

>From what I can tell of the Nova bugs both are issues. Honestly, it
would probably be really telling to setup a test env with 10s token
timeouts and see how crazy it broke. I expect that our expiration logic,
and how our components react to it, is actually a lot less coherent than
we believe.

	-Sean

-- 
Sean Dague
http://dague.net



More information about the OpenStack-dev mailing list