[openstack-dev] [kolla] the alternative of log processing tool

Michał Jastrzębski inc007 at gmail.com
Sun Nov 27 13:46:25 UTC 2016


Hey,

I am also working with Snap community to enable log forwarding with it [1].
Snap is super lightweight and additional benefit of this solution
would be that it can also handle monitoring, which was it's initial
role. One service to handle both would be elegant. I'll keep you
posted but let's not throw away this idea just yet.

Cheers,
Michal

[1] https://github.com/intelsdi-x/snap

On 26 November 2016 at 23:55, Jeffrey Zhang <zhang.lei.fly at gmail.com> wrote:
> Heka is marked deprecated in Kolla during Newton cycle[0]. And Now we have a
> blueprint for this[1]. Two alternatives, fluentd[3] and Filebeat.
>
> For Filebeat, it is just a replacement of logstash-forward[2]. It is not
> intent
> to replace the Logstash at all.
>
>> Filebeat is based on the Logstash Forwarder source code and replaces
>> Logstash
>> Forwarder as the method to use for tailing log files and forwarding them
>> to
>> Logstash.
>
> Fillebeat is a log transport tool rather than log processing too. I do not
> treat it as an alternative at all.
>
> To be honest, I'd like back to Logstash, and Logstash 5.x is released with
> high
> performance improvement[4].
>
>>  In our performance testing, we've seen consistent throughput increases
>>  across multiple configurations. In some cases, we observed up to 75%
>>  increase in events processed through Logstash.
>
> another benefit to using Logstash is the whole ELK stack is maintained by
> one
> community/company. It is well tested and easy to upgrade the whole stack at
> the
> same time. Using other tools may force us on certain elasticsearch release.
>
> So, I think we have to alternative tools.
>
> * Fluentd
> * Logstash
>
> IMO, we need to make the decision and at least prepare the migration
> solution now.
>
> [1] https://blueprints.launchpad.net/kolla/+spec/heka-deprecation
> [2]
> https://www.elastic.co/guide/en/beats/filebeat/current/migrating-from-logstash-forwarder.html
> [3] http://www.fluentd.org/
> [4] https://www.elastic.co/blog/logstash-5-0-0-released
>
> --
> Regards,
> Jeffrey Zhang
> Blog: http://xcodest.me
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



More information about the OpenStack-dev mailing list