[openstack-dev] [keystone][all] Incorporating performance feedback into the review process

Matt Fischer matt at mattfischer.com
Fri Jun 3 19:53:16 UTC 2016

On Fri, Jun 3, 2016 at 1:35 PM, Lance Bragstad <lbragstad at gmail.com> wrote:

> Hey all,
> I have been curious about impact of providing performance feedback as part
> of the review process. From what I understand, keystone used to have a
> performance job that would run against proposed patches (I've only heard
> about it so someone else will have to keep me honest about its timeframe),
> but it sounds like it wasn't valued.
> I think revisiting this topic is valuable, but it raises a series of
> questions.
> Initially it probably only makes sense to test a reasonable set of
> defaults. What do we want these defaults to be? Should they be determined
> by DevStack, openstack-ansible, or something else?
> What does the performance test criteria look like and where does it live?
> Does it just consist of running tempest?

Keystone especially has some calls that are used 1000x or more relative to
others and so I'd be more concerned about them. For me this is token
validation #1 and token creation #2. Tempest checks them of course but
might be too coarse? There are token benchmarks like the ones Dolph and I
use, they are don't mimic a real work flow.  Something to consider.

> From a contributor and reviewer perspective, it would be nice to have the
> ability to compare performance results across patch sets. I understand that
> keeping all performance results for every patch for an extended period of
> time is unrealistic. Maybe we take a daily performance snapshot against
> master and use that to map performance patterns over time?

Having some time series data captured would be super useful. Could we have
daily charts stored indefinitely?

> Have any other projects implemented a similar workflow?
> I'm open to suggestions and discussions because I can't imagine there
> aren't other folks out there interested in this type of pre-merge data
> points.
> Thanks!
> Lance
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160603/d711f395/attachment.html>

More information about the OpenStack-dev mailing list