[openstack-dev] [kolla] the alternative of log processing tool

Jeffrey Zhang zhang.lei.fly at gmail.com
Sun Nov 27 16:40:47 UTC 2016


So filebeat is working with Logstash right? We need split the logs into
pieces by using logstash. IMU, Filebeat do not a variety of processing
plugins, like Logstash[0].

[0] https://www.elastic.co/guide/en/logstash/current/filter-plugins.html

On Sun, Nov 27, 2016 at 11:30 PM, Ian Cordasco <sigmavirus24 at gmail.com>
wrote:

> File beat is maintained be elastic and a part of their product line just
> like ELK. It's a fantastic tool and quite flexible given its age and size
> of codebase
>
> On Nov 26, 2016 11:59 PM, "Jeffrey Zhang" <zhang.lei.fly at gmail.com> wrote:
>
>> Heka is marked deprecated in Kolla during Newton cycle[0]. And Now we
>> have a
>> blueprint for this[1]. Two alternatives, fluentd[3] and Filebeat.
>>
>> For Filebeat, it is just a replacement of logstash-forward[2]. It is not
>> intent
>> to replace the Logstash at all.
>>
>> > Filebeat is based on the Logstash Forwarder source code and replaces
>> Logstash
>> > Forwarder as the method to use for tailing log files and forwarding
>> them to
>> > Logstash.
>>
>> Fillebeat is a log transport tool rather than log processing too. I do not
>> treat it as an alternative at all.
>>
>> To be honest, I'd like back to Logstash, and Logstash 5.x is released
>> with high
>> performance improvement[4].
>>
>> >  In our performance testing, we've seen consistent throughput increases
>> >  across multiple configurations. In some cases, we observed up to 75%
>> >  increase in events processed through Logstash.
>>
>> another benefit to using Logstash is the whole ELK stack is maintained by
>> one
>> community/company. It is well tested and easy to upgrade the whole stack
>> at the
>> same time. Using other tools may force us on certain elasticsearch
>> release.
>>
>> So, I think we have to alternative tools.
>>
>> * Fluentd
>> * Logstash
>>
>> IMO, we need to make the decision and at least prepare the migration
>> solution now.
>>
>> [1] https://blueprints.launchpad.net/kolla/+spec/heka-deprecation
>> [2] https://www.elastic.co/guide/en/beats/filebeat/current/migra
>> ting-from-logstash-forwarder.html
>> [3] http://www.fluentd.org/
>> [4] https://www.elastic.co/blog/logstash-5-0-0-released
>>
>> --
>> Regards,
>> Jeffrey Zhang
>> Blog: http://xcodest.me
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Regards,
Jeffrey Zhang
Blog: http://xcodest.me
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161128/d1a039dc/attachment.html>


More information about the OpenStack-dev mailing list