[Openstack] [Keystone] performance issues after havana upgrade

Morgan Fainberg m at metacloud.com
Mon Jan 13 17:25:07 UTC 2014


Hi Tim,

The change is being proposed directly to stabe/havana.  We have an alternative implementation for Icehouse as we are refactoring the entire key-value-store system and making memcache a version of that new implementation.

Cheers,
Morgan
On January 12, 2014 at 10:14:49, Tim Bell (tim.bell at cern.ch) wrote:

Can we tag this patch for backporting to Havana stable ?  

We're starting work for the CERN upgrade and this looks like a very useful patch to be part of the standard Havana offering.  

Tim  

> -----Original Message-----  
> From: Jonathan Proulx [mailto:jon at jonproulx.com]  
> Sent: 12 January 2014 18:32  
> To: Morgan Fainberg  
> Cc: openstack at lists.openstack.org  
> Subject: Re: [Openstack] [Keystone] performance issues after havana upgrade  
>  
> puzzling side effect?  
>  
> I just made a small change to neutron.conf (adjusted a default quota) and restarted neutron-server, now neutron (but not other services)  
> is  
> spweing:  
>  
> Invalid user token - rejecting request  
>  
> (quite possibly only from dashboard requests CLI seems to work). I've tried restarting keystone (in both wsgi and eventlet modes),  
> restarting neutron-server w/ reverted config and flushing/restarting memcached in various combinations.  
>  
> I don't really see how restarting neutron-server could confuse token validation...  
>  
>  
> On Sun, Jan 12, 2014 at 10:38 AM, Morgan Fainberg <morgan at metacloud.com> wrote:  
> > Thanks for confirming this! It also validates my new logic going into  
> > icehouse (I might have had some ulterior motives here, or not so  
> > ulterior as the case may be). I'll make sure we resolve the test  
> > issues (unrelated to the patch) and get it into the Havana tree so you  
> > don't need to maintain it outside of the releases.  
> >  
> > Cheers,  
> > Morgan  
> >  
> > Sent from my tablet-like-device  
> >  
> >> On Jan 11, 2014, at 11:01 PM, Jonathan Proulx <jon at jonproulx.com> wrote:  
> >>  
> >>> On Sat, Jan 11, 2014 at 10:57 PM, Morgan Fainberg <m at metacloud.com> wrote:  
> >>> Sounds good! Just remember that prior to the fix I posted there,  
> >>> for each token in the user's index, it incurred a round-trip to  
> >>> memcached to validate the token wasn't expired. This change makes  
> >>> it so that there are significantly less trips from keystone to memcached.  
> >>>  
> >>> If this doesn't 100% solve the issue, we should start digging  
> >>> further into what is going on, but I am confident this will (at the  
> >>> very least) help a reasonable amount.  
> >>  
> >> You sir are a miracle worker, my hat is off!  
> >>  
> >> The responsiveness of everything is better than it's ever been, my  
> >> users will think this is the best feature the upgrade.  
> >>  
> >> For example earlier today I managed to launch 10 VMs in parallel,  
> >> eventually, I'd guess on the order of 5-10min. One of my usual  
> >> acceptance tests is being able to launch 100 VMs in that time. Just  
> >> now Iaunched 100 in <2min from request until they'd all been  
> >> provisioned and were booting. Now there's too many moving pieces and  
> >> too few experimental samples to make any publishable claims, but your  
> >> patch is the only thing that changed.  
> >>  
> >> Thanks,  
> >> -Jon  
>  
> _______________________________________________  
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack  
> Post to : openstack at lists.openstack.org  
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140113/d947af8d/attachment.html>


More information about the Openstack mailing list