[openstack-dev] [oslo][messaging][zmq] Discussion on zmq driver design issues
ozamiatin at mirantis.com
Tue Mar 10 13:08:17 UTC 2015
Thanks a lot for your comments.
On 06.03.15 06:21, Eric Windisch wrote:
> On Wed, Mar 4, 2015 at 12:10 PM, ozamiatin <ozamiatin at mirantis.com
> <mailto:ozamiatin at mirantis.com>> wrote:
> By this e-mail I'd like to start a discussion about current zmq
> driver internal design problems I've found out.
> I wish to collect here all proposals and known issues. I hope this
> discussion will be continued on Liberty design summit.
> And hope it will drive our further zmq driver development efforts.
> ZMQ Driver issues list (I address all issues with # and references
> are in ):
> 1. ZMQContext per socket (blocker is neutron improper usage of
> messaging via fork) 
> 2. Too many different contexts.
> We have InternalContext used for ZmqProxy, RPCContext used in
> ZmqReactor, and ZmqListener.
> There is also zmq.Context which is zmq API entity. We need to
> consider a possibility to unify their usage over inheritance
> (maybe stick to RPCContext)
> or to hide them as internal entities in their modules (see
> refactoring #6)
> The code, when I abandoned it, was moving toward fixing these issues,
> but for backwards compatibility was doing so in a staged fashion
> across the stable releases.
> I agree it's pretty bad. Fixing this now, with the driver in a less
> stable state should be easier, as maintaining compatibility is of less
> 3. Topic related code everywhere. We have no topic entity. It is
> all string operations.
> We need some topics management entity and topic itself as an
> entity (not a string).
> It causes issues like , . (I'm already working on it).
> There was a spec related .
> Good! It's ugly. I had proposed a patch at one point, but I believe
> the decision was that it was better and cleaner to move toward the
> oslo.messaging abstraction as we solve the topic issue. Now that
> oslo.messaging exists, I agree it's well past time to fix this
> particular ugliness.
> 4. Manual implementation of messaging patterns.
> Now we can observe poor usage of zmq features in zmq driver.
> Almost everything is implemented over PUSH/PULL.
> 4.1 Manual polling - use zmq.Poller (listening and replying
> for multiple sockets)
> 4.2 Manual request/reply implementation for call .
> Using of REQ/REP (ROUTER/DEALER) socket solves many
> issues. A lot of code may be reduced.
> 4.3 Timeouts waiting
> There are very specific reasons for the use of PUSH/PULL. I'm firmly
> of the belief that it's the only viable solution for an OpenStack RPC
> driver. This has to do with how asynchronous programming in Python is
> performed, with how edge-triggered versus level-triggered events are
> processed, and general state management for REQ/REP sockets.
> I could be proven wrong, but I burned quite a bit of time in the
> beginning of the ZMQ effort looking at REQ/REP before realizing that
> PUSH/PULL was the only reasonable solution. Granted, this was over 3
> years ago, so I would not be too surprised if my assumptions are no
> longer valid.
I agree that REQ/REP is very limited because of their synchronous nature
and 1-to-1 connection possibility.
But there are ROUTER/DEALER proxy sockets recommended to use with
REQ/REP to compose 1-to-N and N-to-N asynchronous patterns.
I'm in research of that now and I didn't finally decide yet. When
everything will be clear for me I'll come with a spec on that.
> 5. Add possibility to work without eventlet . #4.1 is also
> related here, we can reuse many of the implemented solutions
> like zmq.Poller over asynchronous sockets in one separate
> thread (instead of spawning on each new socket).
> I will update the spec  on that.
> Great. This was one of the motivations behind oslo.messaging and it
> would be great to see this come to fruition.
> 6. Put all zmq driver related stuff (matchmakers, most classes
> from zmq_impl) into a separate package.
> Don't keep all classes (ZmqClient, ZmqProxy, Topics management,
> ZmqListener, ZmqSocket, ZmqReactor)
> in one impl_zmq.py module.
> Seems fine. In fact, I think a lot of code could be shared with an
> AMQP v1 driver...
I'll check what can be shared. Actually I didn't yet dig into AMQP v1
> 7. Need more technical documentation on the driver like .
> I'm willing to prepare a current driver architecture overview
> with some graphics UML charts, and to continue discuss the driver
> Documentation has always been a sore point. +2
> Eric Windisch
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev