[openstack-dev] [kolla] the alternative of log processing tool

Jeffrey Zhang zhang.lei.fly at gmail.com
Sun Nov 27 05:55:29 UTC 2016


Heka is marked deprecated in Kolla during Newton cycle[0]. And Now we have a
blueprint for this[1]. Two alternatives, fluentd[3] and Filebeat.

For Filebeat, it is just a replacement of logstash-forward[2]. It is not
intent
to replace the Logstash at all.

> Filebeat is based on the Logstash Forwarder source code and replaces
Logstash
> Forwarder as the method to use for tailing log files and forwarding them
to
> Logstash.

Fillebeat is a log transport tool rather than log processing too. I do not
treat it as an alternative at all.

To be honest, I'd like back to Logstash, and Logstash 5.x is released with
high
performance improvement[4].

>  In our performance testing, we've seen consistent throughput increases
>  across multiple configurations. In some cases, we observed up to 75%
>  increase in events processed through Logstash.

another benefit to using Logstash is the whole ELK stack is maintained by
one
community/company. It is well tested and easy to upgrade the whole stack at
the
same time. Using other tools may force us on certain elasticsearch release.

So, I think we have to alternative tools.

* Fluentd
* Logstash

IMO, we need to make the decision and at least prepare the migration
solution now.

[1] https://blueprints.launchpad.net/kolla/+spec/heka-deprecation
[2]
https://www.elastic.co/guide/en/beats/filebeat/current/migrating-from-logstash-forwarder.html
[3] http://www.fluentd.org/
[4] https://www.elastic.co/blog/logstash-5-0-0-released

-- 
Regards,
Jeffrey Zhang
Blog: http://xcodest.me
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161127/fae905d7/attachment.html>


More information about the OpenStack-dev mailing list