[openstack-dev] [oslo][messaging] Further improvements and refactoring
flavio at redhat.com
Tue Jun 10 12:06:49 UTC 2014
On 10/06/14 15:03 +0400, Dina Belova wrote:
>Oslo.messaging is future of how different OpenStack components communicate with
>each other, and really I’d love to start discussion about how we can make this
>library even better then it’s now and how can we refactor it make more
>As we all remember, oslo.messaging was initially inspired to be created as a
>logical continuation of nova.rpc - as a separated library, with lots of
>transports supported, etc. That’s why oslo.messaging inherited not only
>advantages of now did the nova.rpc work (and it were lots of them), but also
>some architectural decisions that currently sometimes lead to the performance
>issues (we met some of them while Ceilometer performance testing  during the
>For instance, simple testing messaging server (with connection pool and
>eventlet) can process 700 messages per second. The same functionality
>implemented using plain kombu (without connection pool and eventlet) driver is
>processing ten times more - 7000-8000 messages per second.
>So we have the following suggestions about how we may make this process better
>and quicker (and really I’d love to collect your feedback, folks):
>1) Currently we have main loop running in the Executor class, and I guess it’ll
>be much better to move it to the Server class, as it’ll make relationship
>between the classes easier and will leave Executor only one task - process the
>message and that’s it (in blocking or eventlet mode). Moreover, this will make
>further refactoring much easier.
To some extent, the executors are part of the server class since the
later is the one actually controlling them. If I understood your
proposal, the server class would implement the event loop, which means
we would have an EventletServer / BlockingServer, right?
If what I said is what you meant, then I disagree. Executors keep the
eventloop isolated from other parts of the library and this is really
important for us. One of the reason is to easily support multiple
python versions - by having different event loops.
Is my assumption correct? Could you elaborate more?
>2) Some of the drivers implementations (such as impl_rabbit and impl_qpid, for
>instance) are full of useless separated classes that in reality might be
>included to other ones. There are already some changes making the whole
>structure easier , and after the 1st issue will be solved Dispatcher and
>Listener also will be able to be refactored.
This was done on purpose. The idea was to focus on backwards
compatibility rather than cleaning up/improving the drivers. That
said, sounds like those drivers could user some clean up. However, I
think we should first extend the test suite a bit more before hacking
the existing drivers.
>3) If we’ll separate RPC functionality and messaging functionality it’ll make
>code base clean and easily reused.
What do you mean with this?
>4) Connection pool can be refactored to implement more efficient connection
Please, elaborate. What changes do you envision?
As Dims suggested, I think filing some specs for this (and keeping the
proposals separate) would help a lot in understanding what the exact
Glad to know you're looking forward to help improving oslo.messaging.
>Folks, are you ok with such a plan? Alexey Kornienko already started some of
>this work , but really we want to be sure that we chose the correct vector
>of development here.
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 819 bytes
Desc: not available
More information about the OpenStack-dev