[openstack-dev] [oslo.messaging] [zeromq] nova-rpc-zmq-receiver bottleneck

yatin kumbhare yatinkumbhare at gmail.com
Wed Mar 12 13:25:24 UTC 2014

When zeromq is use as rpc-backend, "nova-rpc-zmq-receiver" service needs to
be run on every node.

zmq-receiver receives messages on tcp://*:9501 with socket type PULL and
based on topic-name (which is extracted from received data), it forwards
data to respective local services, over IPC protocol.

While, openstack services, listen/bind on "IPC" socket with socket-type

I see, zmq-receiver as a bottleneck and overhead as per the current design.
1. if this service crashes: communication lost.
2. overhead of running this extra service on every nodes, which just
forward messages as is.

I'm looking forward to, remove zmq-receiver service and enable direct
communication (nova-* and cinder-*) across and within node.

I believe, this will create, zmq experience more seamless.

the communication will change from IPC to zmq TCP socket type for each

like: rpc.cast from scheduler -to - compute would be direct rpc message
passing. no routing through zmq-receiver.

Now, TCP protocol, all services will bind to unique port (port-range could
be, 9501-9510)

from nova.conf, rpc_zmq_matchmaker =

I have put arbitrary ports numbers after the service name.


     "cert:9507": [
     "cinder-scheduler:9508": [
     "cinder-volume:9509": [
     "compute:9501": [
     "conductor:9502": [
     "consoleauth:9503": [
     "network:9504": [
     "scheduler:9506": [
     "zmq_replies:9510": [

Here, the json file would keep track of ports for each services.

Looking forward to seek community feedback on this idea.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140312/29b6b25b/attachment.html>

More information about the OpenStack-dev mailing list