Open Stack

On 03/26/2013 01:34 PM, David Kranz wrote:
> This is without memcache in auth_token. I was trying to find a way 
> past https://bugs.launchpad.net/keystone/+bug/1020127
> which I think I now have. I  would appreciate it if you could validate 
> my comment at the end of that ticket. Here, I just thought that the 
> keystone
> throughput was very low. I know that swift should not be hitting it so 
> hard. If you were referring to using memcache in the keystone server 
> itself then
You can use memcached as an alternate token  back end, but I have no 
reason to thin it would perform any better than SQL.  It was broken 
until fairly recently, too, so I suspect it is not used much in the wild.


> I didn't know you could do that.
>
>  -David
>
>
>
> On 3/26/2013 12:33 PM, Chmouel Boudjnah wrote:
>> this seems to be pretty low, do you have memcaching enabled?
>>
>> On Tue, Mar 26, 2013 at 4:20 PM, David Kranz <david.kranz at qrclab.com> 
>> wrote:
>>> Related to this, I measured that the rate at which keystone (running 
>>> on a
>>> real fairly hefty server) can handle the requests coming from the 
>>> auth_token
>>> middleware (no pki tokens) is about 16/s. That seems pretty low to 
>>> me. Is
>>> there some other keystone performance problem here, or is that not
>>> surprising?
>>>
>>>   -David
>>>
>>>
>>> On 3/24/2013 9:11 PM, Jay Pipes wrote:
>>>> Sure, you could do that, of course. Just like you could use 
>>>> gunicorn or
>>>> some other web server. Just like you could deploy any of the other
>>>> OpenStack services that way.
>>>>
>>>> It would just be nice if one could configure Keystone in the same 
>>>> manner
>>>> that all the other OpenStack services are configured.
>>>>
>>>> -jay
>>>>
>>>> On 03/23/2013 01:19 PM, Joshua Harlow wrote:
>>>>> See: https://github.com/openstack/keystone/tree/master/httpd
>>>>>
>>>>> For example...
>>>>>
>>>>> This lets apache do the multiprocess instead of how nova, glance ...
>>>>> have basically recreated the same mechanism that apache has had for
>>>>> years.
>>>>>
>>>>> Sent from my really tiny device...
>>>>>
>>>>> On Mar 23, 2013, at 10:14 AM, "Joshua Harlow" <harlowja at yahoo-inc.com
>>>>> <mailto:harlowja at yahoo-inc.com>> wrote:
>>>>>
>>>>>> Or I think u can run keystone in wsgi+apache easily, thus getting 
>>>>>> u the
>>>>>> multiprocess support via apache worker processes.
>>>>>>
>>>>>> Sent from my really tiny
>>>>>> device....
>>>>>>
>>>>>> On Mar 22, 2013, at 10:47 AM, "Jay Pipes" <jaypipes at gmail.com
>>>>>> <mailto:jaypipes at gmail.com>>
>>>>>> wrote:
>>>>>>
>>>>>>> Unfortunately, Keystone's WSGI server is only a single process,
>>>>>> with a
>>>>>>> greenthread pool. Unlike Glance, Nova, Cinder, and Swift, which all
>>>>>> use
>>>>>>> multi-process, greenthread-pool-per-process WSGI servers[1],
>>>>>> Keystone
>>>>>>> does it differently[2].
>>>>>>>
>>>>>>> There was a patchset[3] that added
>>>>>> multiprocess support to Keystone, but
>>>>>>> due to objections from termie and
>>>>>> others about it not being necessary,
>>>>>>> it died on the vine. Termie even
>>>>>> noted that Keystone "was designed to be
>>>>>>> run as multiple instances and load
>>>>>> balanced over and [he felt] that
>>>>>>> should be the preferred scaling point".
>>>>>>>
>>>>>>> Because the mysql client connection is C-based, calls to it will be
>>>>>>>
>>>>>> blocking operations on greenthreads within a single process, meaning
>>>>>>> even
>>>>>> if multiple greenthreads are spawned for those 200 incoming
>>>>>>> requests, they
>>>>>> will be processed synchronously.
>>>>>>> The solution is for Keystone to
>>>>>> implement the same multi-processed WSGI
>>>>>>> worker stuff that is in the other
>>>>>> OpenStack projects. Or, diverge from
>>>>>>> the deployment solution of Nova,
>>>>>> Glance, Cinder, and Swift, and manually
>>>>>>> run multiple instances of
>>>>>> keystone, as Termie suggests.
>>>>>>> Best,
>>>>>>> -jay
>>>>>>>
>>>>>>> [1] All pretty much
>>>>>> derived from the original Swift code, with some Oslo
>>>>>>> improvements around
>>>>>> config
>>>>>>> [2] Compare
>>>>>>>
>>>>>> https://github.com/openstack/glance/blob/master/glance/common/wsgi.py 
>>>>>>
>>>>>> with
>>>>>>
>>>>>> https://github.com/openstack/keystone/blob/master/keystone/common/wsgi.py 
>>>>>>
>>>>>> [3] https://review.openstack.org/#/c/7017/
>>>>>>> On 03/21/2013 07:45 AM,
>>>>>> Kanade, Rohan wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I was trying to create 200 users using
>>>>>> the keystone client. All the
>>>>>>>> users are unique and are created on separate
>>>>>> threads which are started
>>>>>>>> at the same time.
>>>>>>>>
>>>>>>>> keystone is handling
>>>>>> each request synchronously , i.e. user 1 is
>>>>>>>> created, then user 2 is
>>>>>> created ...
>>>>>>>> Shouldnt  keystone be running a greenthread for each
>>>>>> request and try to
>>>>>>>> create these users asynchronously?
>>>>>>>> like start
>>>>>> creating user 1 , while handling that request, start creating
>>>>>>>> user 2 or
>>>>>> user n...
>>>>>>>> I have attached the keystone service logs for further
>>>>>> assistance.
>>>>>>>> http://paste.openstack.org/show/34216/
>>>>>>>>
>>>>>>>>
>>>>>> ______________________________________________________________________ 
>>>>>>
>>>>>> Disclaimer:This email and any attachments are sent in strictest
>>>>>> confidence for the sole use of the addressee and may contain legally
>>>>>> privileged, confidential, and proprietary data. If you are not the
>>>>>> intended recipient, please advise the sender by replying promptly to
>>>>>>>> this
>>>>>> email and then delete and destroy this email and any attachments
>>>>>>>> without
>>>>>> any further use, copying or forwarding
>>>>>>>>
>>>>>>>>
>>>>>> _______________________________________________
>>>>>>>> OpenStack-dev mailing
>>>>>> list
>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>> <mailto:OpenStack-dev at lists.openstack.org>
>>>>>>>>
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>
>>>>>> _______________________________________________
>>>>>>> OpenStack-dev mailing
>>>>>> list
>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>> <mailto:OpenStack-dev at lists.openstack.org>
>>>>>>>
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev