<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 07/22/2014 10:44 AM, Sean Dague
wrote:<br>
</div>
<blockquote cite="mid:53CE78BC.3050605@dague.net" type="cite">
<pre wrap="">Honestly, I'm really not sure I see this as a different program, but is
really something that should be folded into the QA program. I feel like
a top level effort like this is going to lead to a lot of duplication in
the data analysis that's currently going on, as well as functionality
for better load driver UX.
-Sean</pre>
</blockquote>
+1<br>
It will also lead to pointless discussions/arguments about which
activities are part of "QA" and which are part of<br>
"Performance and Scalability Testing".<br>
<br>
QA Program mission:<br>
<br>
"
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
<span style="color: rgb(51, 51, 51); font-family: 'Arial Unicode
MS', Arial, sans-serif; font-size: 14px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing: normal;
line-height: 20px; orphans: auto; text-align: start; text-indent:
0px; text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(255, 255, 255); display: inline !important;
float: none;">Develop, maintain, and initiate tools and plans to
ensure the upstream stability and quality of OpenStack, and its
release readiness at any point during the release cycle."</span><br>
<br>
It is hard to see how $subj falls outside of that mission. Of course
rally would continue to have its own repo, review team, etc. as do
tempest and grenade.<br>
<br>
-David<br>
<br>
<blockquote cite="mid:53CE78BC.3050605@dague.net" type="cite">
<pre wrap="">
On 07/21/2014 05:53 PM, Boris Pavlovic wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Stackers and TC,
The Rally contributor team would like to propose a new OpenStack program
with a mission to provide scalability and performance benchmarking, and
code profiling tools for OpenStack components.
We feel we've achieved a critical mass in the Rally project, with an
active, diverse contributor team. The Rally project will be the initial
project in a new proposed "Performance and Scalability" program.
Below, the details on our proposed new program.
Thanks for your consideration,
Boris
[1] <a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/108502/">https://review.openstack.org/#/c/108502/</a>
Official Name
=============
Performance and Scalability
Codename
========
Rally
Scope
=====
Scalability benchmarking, performance analysis, and profiling of
OpenStack components and workloads
Mission
=======
To increase the scalability and performance of OpenStack clouds by:
* defining standard benchmarks
* sharing performance data between operators and developers
* providing transparency of code paths through profiling tools
Maturity
========
* Meeting logs <a class="moz-txt-link-freetext" href="http://eavesdrop.openstack.org/meetings/rally/2014/">http://eavesdrop.openstack.org/meetings/rally/2014/</a>
* IRC channel: #openstack-rally
* Rally performance jobs are in (Cinder, Glance, Keystone & Neutron)
check pipelines.
* > 950 commits over last 10 months
* Large, diverse contributor community
* <a class="moz-txt-link-freetext" href="http://stackalytics.com/?release=juno&metric=commits&project_type=All&module=rally">http://stackalytics.com/?release=juno&metric=commits&project_type=All&module=rally</a>
* <a class="moz-txt-link-freetext" href="http://stackalytics.com/report/contribution/rally/180">http://stackalytics.com/report/contribution/rally/180</a>
* Non official lead of project is Boris Pavlovic
* Official election In progress.
Deliverables
============
Critical deliverables in the Juno cycle are:
* extending Rally Benchmark framework to cover all use cases that are
required by all OpenStack projects
* integrating OSprofiler in all core projects
* increasing functional & unit testing coverage of Rally.
Discussion
==========
One of the major goals of Rally is to make it simple to share results of
standardized benchmarks and experiments between operators and
developers. When an operator needs to verify certain performance
indicators meet some service level agreement, he will be able to run
benchmarks (from Rally) and share with the developer community the
results along with his OpenStack configuration. These benchmark results
will assist developers in diagnosing particular performance and
scalability problems experienced with the operator's configuration.
Another interesting area is Rally & the OpenStack CI process. Currently,
working on performance issues upstream tends to be a more social than
technical process. We can use Rally in the upstream gates to identify
performance regressions and measure improvement in scalability over
time. The use of Rally in the upstream gates will allow a more rigorous,
scientific approach to performance analysis. In the case of an
integrated OSprofiler, it will be possible to get detailed information
about API call flows (e.g. duration of API calls in different services).
_______________________________________________
OpenStack-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OpenStack-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>