<font color="#000000"><font><font face="courier new,monospace">Well, we've conducted some test, but i dont know if it simulate the real use case, actually, the oposite, since the objects that are puted into the cluster, are read no more than twice, the thing is that they are millions.</font></font></font><div>
<font color="#000000"><font><font face="courier new,monospace">So, the test is as follow.</font></font></font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">We are using SWIFT 1.4.8 With Keystone.</font></div>
<div><font face="courier new, monospace">Behind this test as we said there are 6 proxy nodes ( 2 QuadCore owith HT, 96GB of RAM, Memcached with 32GB configured, 2 Bonded 1Gb Nic LACP Mode 4, the interfaces are balanced about 60%-40% )</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><font color="#000000"><font>#1 We put </font></font>30 objects (50K) into a container (private for PUT, public for GET, so no keystone on the GET side), then from 30 different physical hosts acting as clients (96GB fo RAM, 2 Intel QuadCore HT Enabled, 2 Bonded 1Gb Nic) executing "httperf" geting a different image each (HTTP, no SSL) with this command:</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">httperf --server F5_IP_ADDRESS --port 8080 --uri /v1/AUTH_1bf1f1b69a864abb84ed8a1bc82cff21/testCONT/objXX.swf --num-conn 7200 --num-call 1 --rate 50 --timeout 20 --hog -v > swift_$(hostname)</font></div>
<div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">So, each host is getting allways the same object, so doing the math :</font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace">30 clients, 50 req/s per 60 secconds = 90.000 RPM, again, each client is getting ALLWAYS the same object.</font></div><div><font face="courier new, monospace">We monitor this test, and we get throughput peaks of 60.000 RPM at 600ms average response time (with think its a little bit high for lan test), since its true that we are allways getting the same objects for each client, we CANT make the throughput to scale (maybe due to lack of proxy procesing power, but the thing is, the proxies CPU are under the 30% of CPU usage, and Bandwith utilization per NIC peaks 30MB/s).</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">We started with 20 clients and got peaks of 60.000 RPMS, the we went with 30 clients and we cant get through the 60.000 RPMS threshold.</font></div>
<div><font face="courier new, monospace">Its all about proxies ? or we should consider tune the datanode sides</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">----------------</font></div>
<div><font face="courier new, monospace">alejandrito</font></div><div><br></div><br><div class="gmail_quote">On Thu, Oct 25, 2012 at 9:02 AM, Ywang225 <span dir="ltr"><<a href="mailto:ywang225@126.com" target="_blank">ywang225@126.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF"><div>How many disks on each storage node, and what's the model? Normally, small requests performance depends on proxy CPU, but disk model matters, especially for writes. If no bottlenecks on keystone, and disks aren't too bad, I assume over 1000 op/s can archive with one proxy plus 5 storage modes with your pattern.<br>
<br>-ywang</div><div><br>在 2012-10-25，1:56，Alejandro Comisario <<a href="mailto:alejandro.comisario@mercadolibre.com" target="_blank">alejandro.comisario@mercadolibre.com</a>> 写道：<br><br></div><div><div><div>
</div><blockquote type="cite"><div><div><font color="#000000"><font><font face="courier new,monospace">Guys ??</font></font></font></div><font color="#000000"><font><font face="courier new,monospace">Anyone ??</font></font></font><div>
<font color="#000000"><font><font face="courier new,monospace"><br clear="all">
</font></font></font><div><font><b><br></b></font></div><div><font><b><img width="96" height="58"><br></b></font></div><font><b>Alejandro Comisario <br>
#melicloud CloudBuilders</b></font><br>
<font color="#666666"><span style="font-size:6pt;color:gray" lang="ES">Arias 3751, Piso 7 (C1430CRG) <br>
Ciudad de Buenos Aires -
Argentina<br>
Cel: +549(11) 15-3770-1857<br>
Tel : +54(11) 4640-8443</span></font><br>
<br><br><div class="gmail_quote">On Mon, Oct 15, 2012 at 11:59 AM, Kiall Mac Innes <span dir="ltr"><<a href="mailto:kiall@managedit.ie" target="_blank">kiall@managedit.ie</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<p>While I can't answer your question (I've never used swift) - it's worth mentioning many of the openstack folks are en-route/at the design summit.</p>
<p>Also - you might have more luck on the openstack-operators list, rather than the general list.</p>
<p>Kiall</p>
<div class="gmail_quote"><div><div>On Oct 15, 2012 2:57 PM, "Alejandro Comisario" <<a href="mailto:alejandro.comisario@mercadolibre.com" target="_blank">alejandro.comisario@mercadolibre.com</a>> wrote:<br type="attribution">
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
<div><font face="courier new, monospace">Its worth to know that the objects in the cluster, are going to be from 200KB the biggest and 50KB the tiniest.</font></div><div><font face="courier new, monospace">Any considerations regarding this ?</font></div>
<font face="courier new, monospace"><div><font face="courier new, monospace"><br></font></div>-----</font><div><font face="courier new, monospace">alejandrito<br></font><br><div class="gmail_quote">On Thu, Oct 11, 2012 at 8:28 PM, Alejandro Comisario <span dir="ltr"><<a href="mailto:alejandro.comisario@mercadolibre.com" target="_blank">alejandro.comisario@mercadolibre.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Hi Stackers !</div><div>This is the thing, today we have a 24 datanodes (3 copies, 90TB usables) each datanode has 2 intel hexacores CPU with HT and 96GB of RAM, and 6 Proxies with the same hardware configuration, using swift 1.4.8 with keystone.</div>
<div>Regarding the networking, each proxy / datanodes has a dual 1Gb nic, bonded in LACP mode 4, each of the proxies are behind an F5 BigIP Load Balancer ( so, no worries over there ).</div><div><br></div><div>Today, we are receiving 5000 RPM ( Requests per Minute ) with 660 RPM per Proxies, i know its low, but now ... with a new product migration, soon ( really soon ) we are expecting to receive about a total of 90.000 RPM average ( 1500 req / s ) with weekly peaks of 200.000 RPM ( 3500 req / s ) to the swift api, witch will be 90% public gets ( no keystone auth ) and 10% authorized PUTS (keystone in the middle, worth to know that we have a 10 keystone vms pool, connected to a 5 nodes galera mysql cluster, so no worries there either ) </div>
<div><br></div><div>So, 3500 req/s divided by 6 proxy nodes doesnt sounds too much, but well, its a number that we cant ignore.</div><div>What do you think about this numbers? does this 6 proxies sounds good, or we should double or triple the proxies ? Does anyone has this size of requests and can share their configs ?</div>
<div><br></div><div>Thanks a lot, hoping to ear from you guys !</div><div><br></div>-----<br>
<div><font face="courier new, monospace">alejandrito</font></div>
</blockquote></div><br></div>
<br></div></div>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div>
</blockquote></div><br></div>
</div></blockquote></div></div><blockquote type="cite"><div><span>_______________________________________________</span><br><span>OpenStack-operators mailing list</span><br><span><a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a></span><br>
<span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a></span><br></div></blockquote></div></blockquote>
</div><br>
</div>