[Openstack] [Keystone] performance issues after havana upgrade

Tim Bell Tim.Bell at cern.ch
Sun Jan 12 18:14:47 UTC 2014


Can we tag this patch for backporting to Havana stable ? 

We're starting work for the CERN upgrade and this looks like a very useful patch to be part of the standard Havana offering.

Tim

> -----Original Message-----
> From: Jonathan Proulx [mailto:jon at jonproulx.com]
> Sent: 12 January 2014 18:32
> To: Morgan Fainberg
> Cc: openstack at lists.openstack.org
> Subject: Re: [Openstack] [Keystone] performance issues after havana upgrade
> 
> puzzling side effect?
> 
> I just made a small change to neutron.conf (adjusted a default quota) and restarted neutron-server, now neutron (but not other services)
> is
> spweing:
> 
> Invalid user token - rejecting request
> 
> (quite possibly only from dashboard requests CLI seems to work).  I've tried restarting keystone (in both wsgi and eventlet modes),
> restarting neutron-server w/ reverted config and flushing/restarting memcached in various combinations.
> 
> I don't really see how restarting neutron-server could confuse token validation...
> 
> 
> On Sun, Jan 12, 2014 at 10:38 AM, Morgan Fainberg <morgan at metacloud.com> wrote:
> > Thanks for confirming this!  It also validates my new logic going into
> > icehouse (I might have had some ulterior motives here, or not so
> > ulterior as the case may be).  I'll make sure we resolve the test
> > issues (unrelated to the patch) and get it into the Havana tree so you
> > don't need to maintain it outside of the releases.
> >
> > Cheers,
> > Morgan
> >
> > Sent from my tablet-like-device
> >
> >> On Jan 11, 2014, at 11:01 PM, Jonathan Proulx <jon at jonproulx.com> wrote:
> >>
> >>> On Sat, Jan 11, 2014 at 10:57 PM, Morgan Fainberg <m at metacloud.com> wrote:
> >>> Sounds good!  Just remember that prior to the fix I posted there,
> >>> for each token in the user's index, it incurred a round-trip to
> >>> memcached to validate the token wasn't expired.  This change makes
> >>> it so that there are significantly less trips from keystone to memcached.
> >>>
> >>> If this doesn't 100% solve the issue, we should start digging
> >>> further into what is going on, but I am confident this will (at the
> >>> very least) help a reasonable amount.
> >>
> >> You sir are a miracle worker, my hat is off!
> >>
> >> The responsiveness of everything is better than it's ever been, my
> >> users will think this is the best feature the upgrade.
> >>
> >> For example earlier today I managed to launch 10 VMs in parallel,
> >> eventually, I'd guess on the order of 5-10min.  One of my usual
> >> acceptance tests is being able to launch 100 VMs in that time.  Just
> >> now Iaunched 100 in <2min from request until they'd all been
> >> provisioned and were booting.  Now there's too many moving pieces and
> >> too few experimental samples to make any publishable claims, but your
> >> patch is the only thing that changed.
> >>
> >> Thanks,
> >> -Jon
> 
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




More information about the Openstack mailing list