Open Stack

________________________________
From: Lance Bragstad [lbragstad at gmail.com]
Sent: Friday, June 03, 2016 1:57 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [keystone][all] Incorporating performance feedback into the review process

Here is a list of focus points as I see them so far:


  *   Dedicated hardware is a requirement in order to achieve somewhat consistent results
  *   Tight loop micro benchmarks
  *   Tests highlighting the performance cases we care about
  *   The ability to determine a sane control
  *   The ability to tests proposed patches, compare them to the control, and leave comments on reviews
  *   Reproducible setup and test runner so that others can run these against a dedicated performance environment
  *   Daily snapshots of performance published publicly (nice to have)

Time series data (think RRD and family) is very small and cheap. The graphs are pretty trivial to generate from the data. The hard part as Morgan pointed out is finding a place to run it.