Open Stack

On Fri, Jun 3, 2016 at 1:35 PM, Lance Bragstad <lbragstad at gmail.com> wrote:

> Hey all,
>
> I have been curious about impact of providing performance feedback as part
> of the review process. From what I understand, keystone used to have a
> performance job that would run against proposed patches (I've only heard
> about it so someone else will have to keep me honest about its timeframe),
> but it sounds like it wasn't valued.
>
> I think revisiting this topic is valuable, but it raises a series of
> questions.
>
> Initially it probably only makes sense to test a reasonable set of
> defaults. What do we want these defaults to be? Should they be determined
> by DevStack, openstack-ansible, or something else?
>
> What does the performance test criteria look like and where does it live?
> Does it just consist of running tempest?
>

Keystone especially has some calls that are used 1000x or more relative to
others and so I'd be more concerned about them. For me this is token
validation #1 and token creation #2. Tempest checks them of course but
might be too coarse? There are token benchmarks like the ones Dolph and I
use, they are don't mimic a real work flow.  Something to consider.



>
> From a contributor and reviewer perspective, it would be nice to have the
> ability to compare performance results across patch sets. I understand that
> keeping all performance results for every patch for an extended period of
> time is unrealistic. Maybe we take a daily performance snapshot against
> master and use that to map performance patterns over time?
>

Having some time series data captured would be super useful. Could we have
daily charts stored indefinitely?



>
> Have any other projects implemented a similar workflow?
>
> I'm open to suggestions and discussions because I can't imagine there
> aren't other folks out there interested in this type of pre-merge data
> points.
>
> Thanks!
>
> Lance
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160603/d711f395/attachment.html>