<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hey Jay,<div><br></div><div>At the summit, timjr (from Yahoo) demonstrated zipkin tracing of the whole kit, although the patches needed to fully enable that down into the lower layers haven't been pushed into the mainline (to my knowledge). WIth the SQL backend of keystone, I'm already aware of some performance areas, but due to internal post-analysis of HTTP requests flowing to our keystone endpoints during testing (basically, scraping and post-process of timing data from an HAproxy instance). (somewhat akin to your #4 below). </div><div><br></div><div>A coworker of mine did drive on some cprofile equivalent profiling of keystone as well - the only quirk to note there is if you'll likely want to disable any eventlet based workers as it makes the profiling data ... messy (the stack's appear to lie). </div><div><br></div><div>- joe</div><div><br><div><div>On Jun 5, 2013, at 9:20 AM, Jay Buffington <<a href="mailto:me@jaybuff.com">me@jaybuff.com</a>> wrote:</div><blockquote type="cite">Hi Everybody,<div><br></div><div>At the summit there was a session on Keystone Performance. The notes are here:</div><div><a href="https://etherpad.openstack.org/havana-keystone-performance">https://etherpad.openstack.org/havana-keystone-performance</a></div>
<div><br></div><div>My group is experiencing performance problems with our openstack installation that we would like to quantify- you can't fix it if you can't measure it. We have some suspicions that the issue is related to keystone (and our use of LDAP). </div>
<div><br></div><div>Before we invest too much time into this I wanted to know if anyone else has done any profiling of keystone or any other openstack components. Some brainstorming ideas we had when considering how to collect metrics were:</div>
<div><br></div><div>1) zipkin - <a href="https://etherpad.openstack.org/zipkin-tracing">https://etherpad.openstack.org/zipkin-tracing</a></div><div>2) python's cprofile </div><div>3) newrelic</div><div>4) logs have request duration time in them- maybe use splunk somehow?</div>
<div><br></div><div>I think we can profile performance during unit tests and tempest tests.</div><div><br></div><div>Once we have some baseline metrics we can turn on/off features like ldap, pki, etc and eventually do performance testing as a CI gate.</div>
<div><br></div><div>Thanks,</div><div>Jay</div>
</blockquote></div><br></div></body></html>