<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 12/16/2013 02:56 AM, Neependra Khare
wrote:<br>
</div>
<blockquote cite="mid:52AEB242.9040309@redhat.com" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div class="moz-cite-prefix">On 12/14/2013 02:04 AM, Adam Young
wrote:<br>
</div>
<blockquote cite="mid:52AB6F56.9020101@redhat.com" type="cite">On
12/13/2013 02:28 PM, Jay Pipes wrote: <br>
<blockquote type="cite">On 12/13/2013 08:14 AM, Neependra Khare
wrote: <br>
<blockquote type="cite">On 12/12/2013 12:00 PM, Neependra
Khare wrote: <br>
<blockquote type="cite">On 12/12/2013 01:11 AM, Adam Young
wrote: <br>
<blockquote type="cite">Can you indicate which is going to
be your first effort? We <br>
(Keystone team) can provide some guidance on how to best
hammer on it. <br>
</blockquote>
Thanks. I am starting by identifying any CPU, Disk,
Memory or <br>
Database bottlenecks. <br>
</blockquote>
I have listed down the methodology I'll be following for
this test:- <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://wiki.openstack.org/wiki/KeystonePerformance#Identify_CPU.2C_Disk.2C_Memory.2C_Database_bottlenecks">https://wiki.openstack.org/wiki/KeystonePerformance#Identify_CPU.2C_Disk.2C_Memory.2C_Database_bottlenecks</a>
<br>
</blockquote>
<br>
Hi Neependra, <br>
<br>
My first suggestion would be to rework the performance
benchmarking work items to have clearer indications regarding
*what are the metrics being tested* in each work item. <br>
<br>
For example, the first work item is "Identify CPU, Disk,
Memory, and Database Bottlenecks". <br>
<br>
The first test case listed is: <br>
<br>
"Test #1, Create users in parallel and look for CPU, disk or
memory bottleneck." <br>
</blockquote>
<br>
Here is a script you can modify. <br>
<br>
adam.younglogic.com/2013/12/load-keystone-user/ <br>
</blockquote>
Would it make a difference if I use<i> keystoneclient.v2_0</i><span
style="color: rgb(51, 51, 51); font-family: 'Arial Unicode MS',
Arial, sans-serif; font-size: 14.399999618530273px; font-style:
normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: 20px; orphans: auto;
text-align: left; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;
background-color: rgb(255, 255, 255); display: inline
!important; float: none;"></span> python module to create users
as mentioned on the wiki:-<br>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
<a moz-do-not-send="true"
href="https://wiki.openstack.org/wiki/KeystonePerformance#Test_.231.2C_Create_users_in_parallel_and_look_for_CPU.2C_disk_or_memory_bottleneck.">https://wiki.openstack.org/wiki/KeystonePerformance#Test_.231.2C_Create_users_in_parallel_and_look_for_CPU.2C_disk_or_memory_bottleneck.</a><br>
</blockquote>
While it shouldn't make a difference, it would be good to make the
tests future proof. But that approach is good, too. <br>
<br>
<br>
<blockquote cite="mid:52AEB242.9040309@redhat.com" type="cite"> <br>
Thanks, <br>
Neependra<br>
<br>
<blockquote cite="mid:52AB6F56.9020101@redhat.com" type="cite"> <br>
<br>
<blockquote type="cite"> <br>
I think that is a bit too big of an initial bite ;) <br>
<br>
Instead, it may be more effective to instead break down the
performance analysis based on the metrics you wish to test and
the relative conclusions you wish your work to generate. <br>
<br>
For example, consider this possible work item: <br>
<br>
"Determine the maximum number of token authentication calls
that can be performed" <br>
<br>
Within that work item, you can then further expand a testing
matrix, like so: <br>
<br>
* Measure the total number of token authentication calls
performed by a single client against a single-process,
Python-only Keystone server <br>
* Measure the total number of token authentication calls
performed by a single client against a multi-process Keystone
server running inside an nginx or Apache container server --
with 2, 4, 8, 16, and 32 pre-forked processes <br>
* Measure the above using increasing numbers of concurrent
clients -- 10, 50, 100, 500, 1000. <br>
<br>
There's, of course, nothing wrong with measuring things like
CPU, disk and I/O performance during tests, however there
should be a clear metric that is being measured for each test.
<br>
<br>
My second suggestion would be to drop the requirement of using
RDO -- or any version of OpenStack for that matter. <br>
<br>
In these kinds of tests, where you are not measuring the
integrated performance of multiple endpoints, but are instead
measuring the performance of a single endpoint (Keystone),
there's no reason, IMHO, to install all of OpenStack.
Installing and serving the Keystone server (and it's various
drivers) is all that is needed. The fewer "balls up in the
air" during a benchmarking session, the fewer side-effects are
around to effect the outcome of the benchmark... <br>
<br>
Best, <br>
-jay <br>
<br>
<br>
<br>
_______________________________________________ <br>
Mailing list: <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
<br>
Post to : <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a>
<br>
Unsubscribe : <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
<br>
</blockquote>
<br>
<br>
_______________________________________________ <br>
Mailing list: <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
<br>
Post to : <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a>
<br>
Unsubscribe : <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
<br>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>