Open Stack

On 16/07/2013, at 1:10 AM, Adam Young <ayoung at redhat.com> wrote:
> On 07/15/2013 04:06 AM, Kieran Spear wrote:
>> Hi all,
>> 
>> I want to backport the fix for the "Token List in Memcache can consume
>> an entire memcache page" bug[1] to Grizzly, but I had a couple of
>> questions:
>> 
>> 1. Why do we need to store the entire token data in the
>> usertoken-<userid> key? This data always seems to be hashed before
>> indexing into the 'token-<tokenid>' keys anyway. The size of the
>> memcache data for a user's token list currently grows by 4k every time
>> a new PKI token is created. It doesn't take long to hit 1MB at this
>> rate even with the above fix.
> Yep. The reason, though, is that we either take a memory/storage hit (store the whole token) or a performance hit (reproduce the token data) and we've gone for the storage hit.

In this case it looks like we're taking a hit from both, since the PKI token "id" from the user token index is retrieved, then hashed and then that key is used to retrieve the token from the "tokens-%s" page anyway.

> 
> 
>> 
>> 2. Every time it creates a new token, Keystone loads each token from
>> the user's token list with a separate memcache call so it can throw it
>> away if it's expired. This seems excessive. Is it anything to worry
>> about? If it just checked the first two tokens you'd get the same
>> effect on a longer time scale.
>> 
>> I guess part of the answer is to decrease our token expiry time, which
>> should mitigate both issues. Failing that we'd consider moving to the
>> SQL backend.
> HOw about doing both?  But if you move to the sql backend, rememeber to periodically clean up the token table, or you will have storage issues there as well.  No silver bullet, I am afraid.

I think we're going to stick with memcache for now (the devil we know :)). With (1) and (2) fixed and the token expiration time tweaked I think memcache will do okay.

Kieran

> 
>> 
>> Cheers,
>> Kieran
>> 
>> [1] https://bugs.launchpad.net/keystone/+bug/1171985
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev