<div dir="ltr"><div class="gmail_default" style="font-family:monospace,monospace;font-size:small">So filebeat is working with Logstash right? We need split the logs into pieces by using logstash. IMU, Filebeat do not a variety of processing plugins, like Logstash[0].</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:small"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:small">[0] <a href="https://www.elastic.co/guide/en/logstash/current/filter-plugins.html">https://www.elastic.co/guide/en/logstash/current/filter-plugins.html</a> </div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Nov 27, 2016 at 11:30 PM, Ian Cordasco <span dir="ltr"><<a href="mailto:sigmavirus24@gmail.com" target="_blank">sigmavirus24@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">File beat is maintained be elastic and a part of their product line just like ELK. It's a fantastic tool and quite flexible given its age and size of codebase</p>
<div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">On Nov 26, 2016 11:59 PM, "Jeffrey Zhang" <<a href="mailto:zhang.lei.fly@gmail.com" target="_blank">zhang.lei.fly@gmail.com</a>> wrote:<br type="attribution"></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr"><div class="gmail_default"><div class="gmail_default"><font face="monospace, monospace">Heka is marked deprecated in Kolla during Newton cycle[0]. And Now we have a</font></div><div class="gmail_default"><font face="monospace, monospace">blueprint for this[1]. Two alternatives, fluentd[3] and Filebeat. </font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">For Filebeat, it is just a replacement of logstash-forward[2]. It is not intent</font></div><div class="gmail_default"><font face="monospace, monospace">to replace the Logstash at all.</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">> Filebeat is based on the Logstash Forwarder source code and replaces Logstash</font></div><div class="gmail_default"><font face="monospace, monospace">> Forwarder as the method to use for tailing log files and forwarding them to</font></div><div class="gmail_default"><font face="monospace, monospace">> Logstash.</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">Fillebeat is a log transport tool rather than log processing too. I do not</font></div><div class="gmail_default"><font face="monospace, monospace">treat it as an alternative at all.</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">To be honest, I'd like back to Logstash, and Logstash 5.x is released with high</font></div><div class="gmail_default"><font face="monospace, monospace">performance improvement[4].</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">> In our performance testing, we've seen consistent throughput increases</font></div><div class="gmail_default"><font face="monospace, monospace">> across multiple configurations. In some cases, we observed up to 75%</font></div><div class="gmail_default"><font face="monospace, monospace">> increase in events processed through Logstash.</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">another benefit to using Logstash is the whole ELK stack is maintained by one</font></div><div class="gmail_default"><font face="monospace, monospace">community/company. It is well tested and easy to upgrade the whole stack at the</font></div><div class="gmail_default"><font face="monospace, monospace">same time. Using other tools may force us on certain elasticsearch release.</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">So, I think we have to alternative tools.</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">* Fluentd</font></div><div class="gmail_default"><font face="monospace, monospace">* Logstash</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">IMO, we need to make the decision and at least prepare the migration solution now.</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">[1] <a href="https://blueprints.launchpad.net/kolla/+spec/heka-deprecation" target="_blank">https://blueprints.launchpad.n<wbr>et/kolla/+spec/heka-deprecatio<wbr>n</a></font></div><div class="gmail_default"><font face="monospace, monospace">[2] <a href="https://www.elastic.co/guide/en/beats/filebeat/current/migrating-from-logstash-forwarder.html" target="_blank">https://www.elastic.co/guide/e<wbr>n/beats/filebeat/current/migra<wbr>ting-from-logstash-forwarder.<wbr>html</a></font></div><div class="gmail_default"><font face="monospace, monospace">[3] <a href="http://www.fluentd.org/" target="_blank">http://www.fluentd.org/</a></font></div><div class="gmail_default"><font face="monospace, monospace">[4] <a href="https://www.elastic.co/blog/logstash-5-0-0-released" target="_blank">https://www.elastic.co/blog/lo<wbr>gstash-5-0-0-released</a></font></div></div><div><br></div>-- <br><div class="m_-7165405374487821804m_-6319430966046126378gmail-m_-5122375912832051114gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><span style="font-size:13px;border-collapse:collapse"><font face="monospace, monospace">Regards,</font></span></div><div><span style="font-size:13px;border-collapse:collapse"><font face="monospace, monospace">Jeffrey Zhang</font></span></div><div><span style="font-family:monospace,monospace;font-size:12.8px">Blog: </span><a href="http://xcodest.me/" style="font-family:monospace,monospace;font-size:12.8px" target="_blank">http://xcodest.me</a><font face="monospace, monospace"><br></font></div></div></div></div></div></div></div></div></div>
</div>
<br></div></div><span class="">______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.op<wbr>enstack.org?subject:unsubscrib<wbr>e</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-dev</a><br>
<br></span></blockquote></div></div>
<br>______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><span style="font-size:13px;border-collapse:collapse"><font face="monospace, monospace">Regards,</font></span></div><div><span style="font-size:13px;border-collapse:collapse"><font face="monospace, monospace">Jeffrey Zhang</font></span></div><div><span style="font-family:monospace,monospace;font-size:12.8px">Blog: </span><a href="http://xcodest.me/" style="font-family:monospace,monospace;font-size:12.8px" target="_blank">http://xcodest.me</a><font face="monospace, monospace"><br></font></div></div></div></div></div></div></div></div></div>
</div>