Open Stack

Hi to the OpenStack development team.  I am a newbie and this is my first
post.  I apologize now if I am posting this to the wrong place or break
some rule and promise to do better in the future.  Now for the reason for
this post...

I have been investigating why RabbitMQ does not scale for Openstack RPCs.  I
haven’t found any information online that explains why this opinion is
widely held.  From the testing I have done, I have come to the conclusion
that the scaling problem might be limited to RPC calls, as opposed to RPC
casts, and that it is mostly due to AMQP usage pattern coming from
Openstack RPC and perhaps not an inherent problem of RabbitMQ, although my
personal opinion is that RabbitMQ is pretty heavy weight in terms of CPU
burn.  The dynamic creation and deletion of an Exchanges and a Queues for
the delivering of RPC responses is the primary culprit for this lack of
scalability.  My data implies, if not proves, that RabbitQM replicates (in
a clustered environment) each creation and deletion of exchanges and queues
and that this replication is a serialization point that prevents
concurrency.  I have found that maximum concurrent RPC rates decrease as
more RabbitMQ hosts are added to the cluster, again due to the increased
replication workload.



My question is whether anyone thinks it is a worthwhile effort to
investigate code changes to the Openstack RPC that would make the RPC
responses flow on static exchanges and queues?  Or is the general opinion
that RabbitMQ is not good for other reasons and effort should be spent on
some other transport?  Or has this been thought of and rejected as too hard?
Or is someone already working on it?  I would like to work on something
worthwhile with regards to Openstack performance and not duplicating any
else’s short term effort.



Ray
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121113/2537f43b/attachment.html>