<font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2">Hi Gordon,<br><br>As suggested I pasted pipeline.yaml content to <font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2">paste.openstack.org</font><br><br>http://paste.openstack.org/show/558688/<br><br><font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2"><span><font size="2">With Best Regards<br> </font><font size="2">Raghunath Dudyala<br> </font><font size="2">Tata Consultancy Services Limited<br> </font><font size="2">Mailto: raghunath.d@tcs.com<br> </font><font size="2">Website: <a href="http://www.tcs.com">http://www.tcs.com</a><br> </font><font size="2">____________________________________________<br> </font><font size="2">Experience certainty.       IT Services<br> </font><font size="2">                    Business Solutions<br> </font><font size="2">                    Consulting<br> </font><font size="2">____________________________________________<br> </font></span><br><br><font color="#990099">-----gordon chung <gord@live.ca> wrote: -----</font><div class="iNotesHistory" style="padding-left:5px;"><div style="padding-right:0px;padding-left:5px;border-left:solid black 2px;">To: "openstack-dev@lists.openstack.org" <openstack-dev@lists.openstack.org><br>From: gordon chung <gord@live.ca><br>Date: 08/16/2016 07:51PM<br>Subject: Re: [openstack-dev] FW: [Ceilometer]:Duplicate messages with Ceilometer Kafka Publisher.<br><br> <!--Notes ACF <meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">--> <p>what does your pipeline.yaml look like? maybe paste it to paste.openstack.org. i imagine it's correct if your udp publishing works as expected.<br> </p> <br> <div class="moz-cite-prefix">On 16/08/16 04:04 AM, Raghunath D wrote:<br> </div> <blockquote cite="mid:OF43416E78.2481B37E-ON65258011.002BB8D2-65258011.002C557A@tcs.com" type="cite"> <font face="Default Sans        Serif,Verdana,Arial,Helvetica,sans-serif" size="2"><span>Hi Simon,<br> <br> I have two openstack setup's one with kilo and other with mitaka.<br> <br> Please find details of kafka version's below:<br> <b>Kilo kafka client library Version:1.3.1<br> Mitaka kafka client library Version:1.3.1<br> Kafka server version on both kilo and mitaka: kafka_2.11-0.9.0.0.tgz</b><br> <br> One observation is on kilo openstack setup I didn't see duplicate message issue,while on mitaka<br> setup I am experiencing this issue.<br> <br> <font size="2">With Best Regards<br> </font><font size="2">Raghunath Dudyala<br> </font><font size="2">Tata Consultancy Services Limited<br> </font><font size="2">Mailto: <a class="moz-txt-link-abbreviated" href="mailto:raghunath.d@tcs.com"> raghunath.d@tcs.com</a><br> </font><font size="2">Website: <a moz-do-not-send="true" href="http://www.tcs.com"> http://www.tcs.com</a><br> </font><font size="2">____________________________________________<br> </font><font size="2">Experience certainty. IT Services<br> </font><font size="2">Business Solutions<br> </font><font size="2">Consulting<br> </font><font size="2">____________________________________________<br> </font></span><br> <br> <font color="#990099">-</font><br> <div class="iNotesHistory" style="padding-left:5px;"> <div style="padding-right:0px;padding-left:5px;border-left:solid            black 2px;"> <b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b> Simon Pasquier [<a class="moz-txt-link-freetext" href="mailto:spasquier@mirantis.com">mailto:spasquier@mirantis.com</a>] <br> <div class="WordSection1"> <p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><b>Sent:</b> Thursday, August 11, 2016 2:34 AM<br> <b>To:</b> OpenStack Development Mailing List (not for usage questions) <a class="moz-txt-link-rfc2396E" href="mailto:openstack-dev@lists.openstack.org"> <openstack-dev@lists.openstack.org></a><br> <b>Subject:</b> Re: [openstack-dev] [Ceilometer]:Duplicate messages with Ceilometer Kafka Publisher.<o:p></o:p></span></p> <p class="MsoNormal"><o:p> </o:p></p> <div> <div> <div> <div> <p class="MsoNormal">Hi<o:p></o:p></p> </div> <p class="MsoNormal">Which version of Kafka do you use?<o:p></o:p></p> </div> <p class="MsoNormal">BR<o:p></o:p></p> </div> <p class="MsoNormal">Simon<o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> <div> <p class="MsoNormal">On Thu, Aug 11, 2016 at 10:13 AM, Raghunath D <<a moz-do-not-send="true" href="mailto:raghunath.d@tcs.com" target="_blank">raghunath.d@tcs.com</a>> wrote:<o:p></o:p></p> <blockquote style="border:none;border-left:solid                    #CCCCCC 1.0pt;padding:0in 0in 0in                    6.0pt;margin-left:4.8pt;margin-right:0in"> <p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Hi ,</span> <br> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">  We are injecting events to our custom plugin in ceilometer.</span> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">  The ceilometer pipeline.yaml  is configured to publish these events over kafka and udp, consuming these samples using kafka and udp clients.</span> <br> <br> <b><span style="font-size:10.0pt;font-family:"Arial",sans-serif">  KAFKA publisher:</span></b> <br> <b><span style="font-size:10.0pt;font-family:"Arial",sans-serif">  -----------------------</span></b> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">  When the events are sent continously ,we can see duplicate msg's are recevied in kafka client.</span> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">  From the log it seems ceilometer kafka publisher is failed to send msg's,but still these msgs </span><br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">  are received by kafka server.So when kafka resends these failed msgs we can see duplicate msg's</span> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">  received in kafka client.</span> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">  Please find the attached log for reference.</span> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">  Is this know issue ?</span> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">  Is their any workaround for this issue ?</span> <br> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <b><u> UDP publisher:</u></b></span> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">    No duplicate msg's issue seen here and it is working as expected.</span> <br> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">        </span><br> <br> <span style="font-size:10.0pt;font-family:"Arial",sans-serif">With Best Regards<br> Raghunath Dudyala<br> Tata Consultancy Services Limited<br> Mailto: <a moz-do-not-send="true" href="mailto:raghunath.d@tcs.com" target="_blank"> raghunath.d@tcs.com</a><br> Website: </span><a moz-do-not-send="true" href="http://www.tcs.com/" target="_blank"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">http://www.tcs.com</span></a><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br> ____________________________________________<br> Experience certainty.        IT Services<br>                        Business Solutions<br>                        Consulting<br> ____________________________________________</span><o:p></o:p></p> <p>=====-----=====-----=====<br> Notice: The information contained in this e-mail<br> message and/or attachments to it may contain <br> confidential or privileged information. If you are <br> not the intended recipient, any dissemination, use, <br> review, distribution, printing or copying of the <br> information contained in this e-mail message <br> and/or attachments to it are strictly prohibited. If <br> you have received this communication in error, <br> please notify us by reply e-mail or telephone and <br> immediately and permanently delete the message <br> and any attachments. Thank you<o:p></o:p></p> <p class="MsoNormal" style="margin-bottom:12.0pt"><br> __________________________________________________________________________<br> OpenStack Development Mailing List (not for usage questions)<br> Unsubscribe: <a moz-do-not-send="true" href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank"> OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br> <a moz-do-not-send="true" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p> </blockquote> </div> <p class="MsoNormal"><o:p> </o:p></p> </div> </div> </div> <br> <br> [attachment "ATT00001.txt" removed by Raghunath D/HYD/TCS]</div> </font><br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <div><font face="Courier New,Courier,monospace" size="2">__________________________________________________________________________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br><a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br></font></div> </blockquote> <br> <div><font face="Courier New,Courier,monospace" size="2">-- <br>gord</font></div> <div><font face="Courier New,Courier,monospace" size="2">__________________________________________________________________________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe<br><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br></font></div></div></div></font> </font>