<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Ray<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">What about discussing your results with rabbimq team (through the rabbimtq mailing list)?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Maybe we are missing something.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">--<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Andrea
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Ray Pekowski [mailto:pekowski@gmail.com]
<br>
<b>Sent:</b> 17 November 2012 00:13<br>
<b>To:</b> OpenStack Development Mailing List<br>
<b>Subject:</b> Re: [openstack-dev] RabbitMQ Scaling<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Fri, Nov 16, 2012 at 3:41 AM, Rosa, Andrea (HP Cloud Services) <<a href="mailto:andrea.rosa@hp.com" target="_blank">andrea.rosa@hp.com</a>> wrote:<o:p></o:p></p>
<p class="MsoNormal">I am really surprised that you have the same results. The configuration with RAM nodes should have an impact on the performance.<br>
I have some other questions:<br>
- are you creating a new connection for each call or do you have a connectionPool?<o:p></o:p></p>
<div>
<p class="MsoNormal"><br>
I just turned on the OpenStack DEBUG level logging and what I see from my client RPC load generator is only one "Pool creating new connection" message, so it appears that it is using a connectionPool for all other requests. Since I add load by starting a new
client RPC load generator every 60 seconds, a new pool is created every 60 seconds.<br>
<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">- do you have multiple producers and a single consumer? I mean all RPC are requests are for a single queue? If so are you using a specific prefetch value? Are you using auto_ack?<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><br>
Every 10 load generators go to a single consumer (simulated service). Every group of 10 load generators goes to a different consumer (simulated service), but the performance really is bad from the very first load generator (when compared to RPC casts). For
example, the first load generator in the RPC call case only achieves 29 RPCs/sec while in the cast case, it achieves 240 RPCs/sec.<br>
<br>
I don’t know about prefetch or auto_ack. Since I am using the OpenStack RPC abstraction it is whatever OpenStack is using for these settings. It might not be significant, since I am seeing much better performance with casts than calls and the casts use static
queues and exchanges, by static I mean they are created only once.<br>
<br>
More evidence to this being due to serialized replication on the dynamically created queues and exchanges is the following table. The numbers are for the time between the prior action and the start of the action listed and they are listed in real order of
occurrence. The "declares", bind and close (implies that close destroys the "auto delete" exchange and queue) are all significantly longer. This is from a wireshark trace of a single producer and a single consumer from the RabbbitMQ server. RabbitMQ is
taking significantly longer to do something in the clustered case and it seems likely that that something is most likely the replication of the creation and destruction of the queue and exchange.<br>
<br>
<span style="font-family:"Courier New"">No cluster Cluster of 3 AMQP Action<br>
(in ms) (in ms)<br>
0.647 1.704 Type: Method (1) Channel: 1 Class: Channel (20) Method: Open (10)<br>
0.451 0.468 Type: Method (1) Channel: 1 Class: Channel (20) Method: Open-Ok (11)<br>
1.484 1.459 Type: Method (1) Channel: 1 Class: Exchange (40) Method: Declare (10) Exchange: XXXXXXXXXX Type: direct
<br>
0.431 <b>2.612</b> Type: Method (1) Channel: 1 Class: Exchange (40) Method: Declare-Ok (11)<br>
0.612 0.612 Type: Method (1) Channel: 1 Class: Queue (50) Method: Declare (10) Queue: XXXXXXXXXX<br>
0.86 <b>4.369</b> Type: Method (1) Channel: 1 Class: Queue (50) Method: Declare-Ok (11) Queue: XXXXXXXXXX<br>
0.64 0.723 Type: Method (1) Channel: 1 Class: Queue (50) Method: Bind (20) Queue: XXXXXXXXXX Exchange: XXXXXXXXXX Routing-Key: XXXXXXXXXX<br>
0.622 <b>3.34</b> Type: Method (1) Channel: 1 Class: Queue (50) Method: Bind-Ok (21)<br>
0.758 0.731 Type: Method (1) Channel: 1 Class: Exchange (40) Method: Declare (10) Exchange: nova Type: topic<br>
0.193 0.194 Type: Method (1) Channel: 1 Class: Exchange (40) Method: Declare-Ok (11)<br>
0.864 0.886 Type: Method (1) Channel: 1 Class: Basic (60) Method: Publish (40) Exchange: nova Routing-Key: perfsvc1<br>
0.029 0.034 Type: Content header (2) Channel: 1<br>
0.067 0.068 Type: Method (1) Channel: 1 Class: Basic (60) Method: Consume (20) Queue: XXXXXXXXXX<br>
0.607 1.186 Type: Method (1) Channel: 1 Class: Basic (60) Method: Consume-Ok (21)<br>
8.408 <b>9.461</b> Type: Method (1) Channel: 1 Class: Channel (20) Method: Close (40)<br>
1.8 10.47 Type: Method (1) Channel: 1 Class: Channel (20) Method: Close-Ok (41)</span><br>
<br>
Ray<o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>