Open Stack

Hello, stackers!


Oslo.messaging is future of how different OpenStack components communicate
with each other, and really I’d love to start discussion about how we can
make this library even better then it’s now and how can we refactor it make
more production-ready.


As we all remember, oslo.messaging was initially inspired to be created as
a logical continuation of nova.rpc - as a separated library, with lots of
transports supported, etc. That’s why oslo.messaging inherited not only
advantages of now did the nova.rpc work (and it were lots of them), but
also some architectural decisions that currently sometimes lead to the
performance issues (we met some of them while Ceilometer performance
testing [1] during the Icehouse).


For instance, simple testing messaging server (with connection pool and
eventlet) can process 700 messages per second. The same functionality
implemented using plain kombu (without connection pool and eventlet)
driver is processing ten times more - 7000-8000 messages per second.


So we have the following suggestions about how we may make this process
better and quicker (and really I’d love to collect your feedback, folks):


1) Currently we have main loop running in the Executor class, and I guess
it’ll be much better to move it to the Server class, as it’ll make
relationship between the classes easier and will leave Executor only one
task - process the message and that’s it (in blocking or eventlet mode).
Moreover, this will make further refactoring much easier.

2) Some of the drivers implementations (such as impl_rabbit and impl_qpid,
for instance) are full of useless separated classes that in reality might
be included to other ones. There are already some changes making the whole
structure easier [2], and after the 1st issue will be solved Dispatcher and
Listener also will be able to be refactored.

3) If we’ll separate RPC functionality and messaging functionality it’ll
make code base clean and easily reused.

4) Connection pool can be refactored to implement more efficient connection
reusage.


Folks, are you ok with such a plan? Alexey Kornienko already started some
of this work [2], but really we want to be sure that we chose the correct
vector of development here.


Thanks!


[1]
https://docs.google.com/document/d/1ARpKiYW2WN94JloG0prNcLjMeom-ySVhe8fvjXG_uRU/edit?usp=sharing

[2]
https://review.openstack.org/#/q/status:open+owner:akornienko+project:openstack/oslo.messaging,n,z

Best regards,

Dina Belova

Software Engineer

Mirantis Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140610/b66e35e0/attachment.html>