<div dir="ltr">Hi Mike,<div><br></div><div>Thanks for your feedback.</div><div><br></div><div>I'm not aware on the details of ceilometer messaging.</div><div><br></div><div>Would please point out on "<span style="font-size:13px;font-family:arial,sans-serif">messaging </span><span style="font-size:13px;font-family:arial,sans-serif">behaviors that are desirable for ceilometer currently and possibly other things in the future"?</span></div>
<div><br></div><div>This will help me in evaluating my idea further.<br></div><div><br></div><div>Regards,</div><div>Yatin</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Mar 15, 2014 at 12:15 AM, Mike Wilson <span dir="ltr"><<a href="mailto:geekinutah@gmail.com" target="_blank">geekinutah@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Yatin,<div><br></div><div>I'm glad you are thinking about the drawbacks of the zmq-receiver causes, I want to give you a reason to keep the zmq-receiver and get your feedback. The way I think about the zmq-receiver is a tiny little mini-broker that exists separate from any other OpenStack service. As such, it's implementation can be augmented to support store-and-forward and possibly other messaging behaviors that are desirable for ceilometer currently and possibly other things in the future. Integrating the receiver into each service is going to remove its independence and black box nature and give it all the bugs and quirks of any project it gets lumped in with. I would prefer that we continue to improve zmq-receiver to overcome the tough parts. Either that or find a good replacement and use that. An example of a possible replacement might be the qpid dispatch router[1], although this guy explicitly wants to avoid any store and forward behaviors. Of course, dispatch router is going to be tied to qpid, I just wanted to give an example of something with similar functionality.</div>
<div><br></div><div>-Mike</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On Thu, Mar 13, 2014 at 11:36 AM, yatin kumbhare <span dir="ltr"><<a href="mailto:yatinkumbhare@gmail.com" target="_blank">yatinkumbhare@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr"><div class="gmail_quote">Hello Folks,</div><div class="gmail_quote"><br>
<div dir="ltr"><div>When zeromq is use as rpc-backend, "nova-rpc-zmq-receiver" service needs to be run on every node.</div>
<div>
<br></div><div>zmq-receiver receives messages on tcp://*:9501 with socket type PULL and</div>
<div>based on topic-name (which is extracted from received data), it forwards data to respective local services, over IPC protocol.</div><div><br></div><div>While, openstack services, listen/bind on "IPC" socket with socket-type PULL.</div>
<div><br></div><div>I see, zmq-receiver as a bottleneck and overhead as per the current design. </div><div>1. if this service crashes: communication lost.</div><div>2. overhead of running this extra service on every nodes, which just forward messages as is.</div>
<div><br></div><div><br></div><div>I'm looking forward to, remove zmq-receiver service and enable direct communication (nova-* and cinder-*) across and within node.</div><div><br></div><div>I believe, this will create, zmq experience more seamless.</div>
<div><br></div><div>the communication will change from IPC to zmq TCP socket type for each service.</div><div><br></div><div>like: rpc.cast from scheduler -to - compute would be direct rpc message passing. no routing through zmq-receiver.</div>
<div><br></div><div>Now, TCP protocol, all services will bind to unique port (port-range could be, 9501-9510)</div><div><br></div><div>from nova.conf, rpc_zmq_matchmaker = nova.openstack.common.rpc.matchmaker_ring.MatchMakerRing.</div>
<div><br></div><div>I have put arbitrary ports numbers after the service name.</div><div><br></div><div>file:///etc/oslo/matchmaker_ring.json</div><div><br></div><div> {</div><div> "cert:9507": [</div><div>
"controller"</div><div> ],</div><div> "cinder-scheduler:9508": [</div><div> "controller"</div><div> ],</div><div> "cinder-volume:9509": [</div><div>
"controller"</div><div> ],</div><div> "compute:9501": [</div><div> "controller","computenodex"</div><div> ],</div><div> "conductor:9502": [</div>
<div> "controller"</div><div> ],</div><div> "consoleauth:9503": [</div><div> "controller"</div><div> ],</div><div> "network:9504": [</div><div> "controller","computenodex"</div>
<div> ],</div><div> "scheduler:9506": [</div><div> "controller"</div><div> ],</div><div> "zmq_replies:9510": [</div><div> "controller","computenodex"</div>
<div> ]</div><div> }</div><div> </div><div>Here, the json file would keep track of ports for each services.</div><div><br></div><div>Looking forward to seek community feedback on this idea.</div><div><br></div><div><br>
</div><div>Regards,</div><div>Yatin</div></div>
</div><br></div>
<br></div></div>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>