<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 28, 2015 at 11:39 AM, Flavio Percoco <span dir="ltr"><<a href="mailto:flavio@redhat.com" target="_blank">flavio@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class=""><div class="h5">On 28/01/15 10:23 +0200, Denis Makogon wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<br>
On Tue, Jan 27, 2015 at 10:26 PM, Gordon Sim <<a href="mailto:gsim@redhat.com" target="_blank">gsim@redhat.com</a>> wrote:<br>
<br>
On 01/27/2015 06:31 PM, Doug Hellmann wrote:<br>
On Tue, Jan 27, 2015, at 12:28 PM, Denis Makogon wrote:<br>
I'd like to build tool that would be able to profile messaging over<br>
various deployments. This "tool" would give me an ability to<br>
compare<br>
results of performance testing produced by native tools and<br>
oslo.messaging-based tool, eventually it would lead us into digging<br>
into<br>
code and trying to figure out where "bad things" are happening<br>
(that's<br>
the<br>
actual place where we would need to profile messaging code).<br>
Correct me<br>
if<br>
i'm wrong.<br>
<br>
<br>
It would be interesting to have recommendations for deployment of<br>
rabbit<br>
or qpid based on performance testing with oslo.messaging. It would also<br>
be interesting to have recommendations for changes to the<br>
implementation<br>
of oslo.messaging based on performance testing. I'm not sure you want<br>
to<br>
do full-stack testing for the latter, though.<br>
<br>
Either way, I think you would be able to start the testing without any<br>
changes in oslo.messaging.<br>
<br>
I agree. I think the first step is to define what to measure and then<br>
construct an application using olso.messaging that allows the data of<br>
interest to be captured using different drivers and indeed different<br>
configurations of a given driver.<br>
<br>
I wrote a very simple test application to test one aspect that I felt was<br>
important, namely the scalability of the RPC mechanism as you increase the<br>
number of clients and servers involved. The code I used is https://<br>
<a href="http://github.com/grs/ombt" target="_blank">github.com/grs/ombt</a>, its probably stale at the moment, I only link to it as<br>
an example of approach.<br>
<br>
Using that test code I was then able to compare performance in this one<br>
aspect across drivers (the 'rabbit', 'qpid' and new amqp 1.0 based drivers<br>
_ I wanted to try zmq, but couldn't figure out how to get it working at the<br>
time), and for different deployment options using a given driver (amqp 1.0<br>
using qpidd or qpid dispatch router in either standalone or with multiple<br>
connected routers).<br>
<br>
There are of course several other aspects that I think would be important<br>
to explore: notifications, more specific variations in the RPC 'topology'<br>
i.e. number of clients on given server number of servers in single group<br>
etc, and a better tool (or set of tools) would allow all of these to be<br>
explored.<br>
<br>
From my experimentation, I believe the biggest differences in scalability<br>
are going to come not from optimising the code in oslo.messaging so much as<br>
choosing different patterns for communication. Those choices may be<br>
constrained by other aspects as well of course, notably approach to<br>
reliability.<br>
<br>
<br>
<br>
<br>
After couple internal discussions and hours of investigations, i think i've<br>
foung the most applicabale solution <br>
that will accomplish performance testing approach and will eventually be<br>
evaluated as messaging drivers <br>
configuration and AMQP service deployment recommendataion.<br>
<br>
Solution that i've been talking about is already pretty well-known across<br>
OpenStack components - Rally and its scenarios.<br>
Why it would be the best option? Rally scenarios would not touch messaging<br>
core part. Scenarios are gate-able. <br>
Even if we're talking about internal testing, scenarios are very useful in this<br>
case, <br>
since they are something that can be tuned/configured taking into account<br>
environment needs.<br>
<br>
Doug, Gordon, what do you think about bringing scenarios into messaging? <br>
</blockquote>
<br></div></div>
I personally wouldn't mind having them but I'd like us to first<br>
discuss what kind of scenarios we want to test.<br>
<br>
I'm assuming these scenarios would be pure oslo.messaging scenarios<br>
and they won't require any of the openstack services. Therefore, I<br>
guess these scenarios would test things like performance with many </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
consumers, performance with several (a)synchronous calls, etc. What<br>
performance means in this context will have to be discussed as well.<br></blockquote><div><div><br></div><div>Correct, oslo.messaging scenarios would expect to have only AMQP service and nothing else.</div><div>Yes, that's what i've been thinking about. Also, i'd like to share doc that i've found, see [1].</div><div>As i can see it would be more than useful to enable next scenarios:</div><div><ul><li>Single multi-thread publisher (rpc client) against single multi-thread consumer</li><ul><li>using RPC cast/call methods try to measure time between request and response.</li></ul><li>Multiple multi-thread publishers against single multi-thread consumer</li><ul><li>using RPC cast/call methods try to measure time between requests and responses to multiple publishers.</li></ul><li>Multiple multi-thread publishers against multiple multi-thread consumers</li></ul></div><div><br></div><div>[1] <a href="http://agocs.org/blog/2014/08/19/rabbitmq-best-practices-in-go/">http://agocs.org/blog/2014/08/19/rabbitmq-best-practices-in-go/</a></div></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
In addition to the above, it'd be really interesting if we could have<br>
tests for things like reconnects delays, which I think is doable with<br>
Rally. Am I right?<br>
<br></blockquote><div><br></div><div>That's might be the case. But i'd ask Rally team first.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Cheers,<br>
Flavio<div class=""><div class="h5"><br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<br>
______________________________<u></u>______________________________<u></u>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.<u></u>openstack.org?subject:<u></u>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
<br>
<br>
<br>
Kind regards,<br>
Denis M.<br>
<br>
</blockquote>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
______________________________<u></u>______________________________<u></u>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.<u></u>openstack.org?subject:<u></u>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
</blockquote>
<br>
<br></div></div><span class=""><font color="#888888">
-- <br>
@flaper87<br>
Flavio Percoco<br>
</font></span><br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div><div class="gmail_extra">Kind regards,</div><div class="gmail_extra">Denis M.</div><div class="gmail_extra"><br></div></div>