[openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility
Sean Dague
sean at dague.net
Thu Sep 11 11:48:59 UTC 2014
On 09/10/2014 11:55 AM, Steven Hardy wrote:
> On Wed, Sep 10, 2014 at 10:14:32AM -0400, Sean Dague wrote:
>> Going through the untriaged Nova bugs, and there are a few on a similar
>> pattern:
>>
>> Nova operation in progress.... takes a while
>> Crosses keystone token expiration time
>> Timeout thrown
>> Operation fails
>> Terrible 500 error sent back to user
>
> We actually have this exact problem in Heat, which I'm currently trying to
> solve:
>
> https://bugs.launchpad.net/heat/+bug/1306294
>
> Can you clarify, is the issue either:
>
> 1. Create novaclient object with username/password
> 2. Do series of operations via the client object which eventually fail
> after $n operations due to token expiry
>
> or:
>
> 1. Create novaclient object with username/password
> 2. Some really long operation which means token expires in the course of
> the service handling the request, blowing up and 500-ing
>From what I can tell of the Nova bugs both are issues. Honestly, it
would probably be really telling to setup a test env with 10s token
timeouts and see how crazy it broke. I expect that our expiration logic,
and how our components react to it, is actually a lot less coherent than
we believe.
-Sean
--
Sean Dague
http://dague.net
More information about the OpenStack-dev
mailing list