[openstack-dev] [oslo][messaging][zmq] Discussion on zmq driver design issues

Li Ma skywalker.nick at gmail.com
Sat Mar 21 07:53:50 UTC 2015

On Tue, Mar 10, 2015 at 8:14 PM, ozamiatin <ozamiatin at mirantis.com> wrote:
> Hi Li Ma,
> Thank you very much for your reply

> It's good to hear you have a living deployment with zmq driver.
> Is there a big divergence between your production and upstream versions
> of the driver? Besides [1] and [2] fixes for redis we have [5] and [6]
> critical multi-backend related issues for using the driver in real-world
> deployment.

Actually there's no such a big divergence between our driver and
upstream version. We didn't refactor it much, but just fixed all the
bugs that we met before and implemented socket reuse mechanism to
greatly improve its performance. For some bugs available, especially
cinder multi-backend and neutron multi-worker, we hacked cinder and
neutron to get rid of these bugs.

I discussed with our cinder developer several times about these
problems you mentioned above. Due to the current architecture, it is
really difficult to fix it in zeromq driver. However, it is very easy
to deal with it in cinder. We have patches on hand, but the
implementation is a little tricky that the upstream may not accept it.
:-( No worry, I'll find out it soon.

By the way, we are discussing about fanout performance and message
persistance. I don't have codes available, but I've got some ideas to
implement it.

>> The only functionality for large-scale deployment that lacks in the
>> current upstream codebase is socket pool scheduling (to provide
>> lifecycle management, including recycle and reuse zeromq sockets). It
>> was done several months ago and we are willing to contribute. I plan
>> to propose a blueprint in the next release.
> Pool, recycle and reuse sounds good for performance.

Yes, actually our implementation is a little ugly and there's no unit
test available. Right now, I'm trying to refactor it and hopefully
I'll submit a spec soon.

> We also need a refactoring of the driver to reduce redundant entities
> or reconsider them (like ZmqClient or InternalContext) and to reduce code
> replications (like with topics).
> There is also some topics management needed.
> Clear code == less bugs == easy understand == easy contribute.
> We need a discussion (with related spec and UMLs) about what the driver
> architecture should be (I'm in progress with that).

+1, cannot agree with you more.

>> 3. ZeroMQ integration
>> I've been working on the integration of ZeroMQ and DevStack for a
>> while and actually it is working right now. I updated the deployment
>> guide [3].
> That's true it works! :)
>> I think it is the time to bring a non-voting gate for ZeroMQ and we
>> can make the functional tests work.
> You can turn it with 'check experimental'. It is broken now.

I'll figure it out soon.

>> 5. ZeroMQ discussion
>> Here I'd like to say sorry for this driver. Due to spare time and
>> timezone, I'm not available for IRC or other meeting or discussions.
>> But if it is possible, should we create a subgroup for ZeroMQ and
>> schedule meetings for it? If we can schedule in advance or at a fixed
>> date & time, I'm in.
> That's great idea
> +1 for zmq subgroup and meetings

I'll open another thread to discuss this topic.

> Subfolder is actually what I mean (python package like '_drivers')
> it should stay in oslo.messaging. Separate package like
> oslo.messaging.zeromq is overkill.
> As Doug proposed we can do consistently to AMQP-driver.

I suggest you go for it right now. It is really important for further
If I submit new codes based upon the current code structure, it will
greatly affect this work in the future.

Best regards,
Li Ma (Nick)
Email: skywalker.nick at gmail.com

More information about the OpenStack-dev mailing list