Open Stack

Hi, Eric

Thanks a lot for your comments.

On 06.03.15 06:21, Eric Windisch wrote:
> On Wed, Mar 4, 2015 at 12:10 PM, ozamiatin <ozamiatin at mirantis.com 
> <mailto:ozamiatin at mirantis.com>> wrote:
>
>     Hi,
>
>     By this e-mail I'd like to start a discussion about current zmq
>     driver internal design problems I've found out.
>     I wish to collect here all proposals and known issues. I hope this
>     discussion will be continued on Liberty design summit.
>     And hope it will drive our further zmq driver development efforts.
>
>     ZMQ Driver issues list (I address all issues with # and references
>     are in []):
>
>     1. ZMQContext per socket (blocker is neutron improper usage of
>     messaging via fork) [3]
>     2. Too many different contexts.
>         We have InternalContext used for ZmqProxy, RPCContext used in
>     ZmqReactor, and ZmqListener.
>         There is also zmq.Context which is zmq API entity. We need to
>     consider a possibility to unify their usage over inheritance
>     (maybe stick to RPCContext)
>         or to hide them as internal entities in their modules (see
>     refactoring #6)
>
>
> The code, when I abandoned it, was moving toward fixing these issues, 
> but for backwards compatibility was doing so in a staged fashion 
> across the stable releases.
>
> I agree it's pretty bad. Fixing this now, with the driver in a less 
> stable state should be easier, as maintaining compatibility is of less 
> importance.
>
>     3. Topic related code everywhere. We have no topic entity. It is
>     all string operations.
>         We need some topics management entity and topic itself as an
>     entity (not a string).
>         It causes issues like [4], [5]. (I'm already working on it).
>         There was a spec related [7].
>
>
> Good! It's ugly. I had proposed a patch at one point, but I believe 
> the decision was that it was better and cleaner to move toward the 
> oslo.messaging abstraction as we solve the topic issue. Now that 
> oslo.messaging exists, I agree it's well past time to fix this 
> particular ugliness.
>
>     4. Manual implementation of messaging patterns.
>        Now we can observe poor usage of zmq features in zmq driver.
>     Almost everything is implemented over PUSH/PULL.
>
>         4.1 Manual polling - use zmq.Poller (listening and replying
>     for multiple sockets)
>         4.2 Manual request/reply implementation for call [1].
>             Using of REQ/REP (ROUTER/DEALER) socket solves many
>     issues. A lot of code may be reduced.
>         4.3 Timeouts waiting
>
>
> There are very specific reasons for the use of PUSH/PULL. I'm firmly 
> of the belief that it's the only viable solution for an OpenStack RPC 
> driver. This has to do with how asynchronous programming in Python is 
> performed, with how edge-triggered versus level-triggered events are 
> processed, and general state management for REQ/REP sockets.
>
> I could be proven wrong, but I burned quite a bit of time in the 
> beginning of the ZMQ effort looking at REQ/REP before realizing that 
> PUSH/PULL was the only reasonable solution. Granted, this was over 3 
> years ago, so I would not be too surprised if my assumptions are no 
> longer valid.
>
I agree that REQ/REP is very limited because of their synchronous nature 
and 1-to-1 connection possibility.
But there are ROUTER/DEALER proxy sockets recommended to use with 
REQ/REP to compose 1-to-N and N-to-N asynchronous patterns.
I'm in research of that now and I didn't finally decide yet. When 
everything will be clear for me I'll come with a spec on that.
>
>     5. Add possibility to work without eventlet [2]. #4.1 is also
>     related here, we can reuse many of the implemented solutions
>        like zmq.Poller over asynchronous sockets in one separate
>     thread (instead of spawning on each new socket).
>        I will update the spec [2] on that.
>
>
> Great. This was one of the motivations behind oslo.messaging and it 
> would be great to see this come to fruition.
>
>     6. Put all zmq driver related stuff (matchmakers, most classes
>     from zmq_impl) into a separate package.
>        Don't keep all classes (ZmqClient, ZmqProxy, Topics management,
>     ZmqListener, ZmqSocket, ZmqReactor)
>        in one impl_zmq.py module.
>
>
> Seems fine. In fact, I think a lot of code could be shared with an 
> AMQP v1 driver...

I'll check what can be shared. Actually I didn't yet dig into AMQP v1 
driver enough.
>
>     7. Need more technical documentation on the driver like [6].
>        I'm willing to prepare a current driver architecture overview
>     with some graphics UML charts, and to continue discuss the driver
>     architecture.
>
>
> Documentation has always been a sore point. +2
> -- 
> Regards,
> Eric Windisch
> ᐧ
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Regards,
Oleksii Zamiatin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150310/31f7b985/attachment.html>