[openstack-dev] [kolla] the alternative of log processing tool

Michał Jastrzębski inc007 at gmail.com
Sun Nov 27 15:09:26 UTC 2016


It's in development

On 27 November 2016 at 08:50, Shake Chen <shake.chen at gmail.com> wrote:
> Hi Michal
>
> the snap seem not support log forward now, I can not find any infomation in
> google .
>
> On Sun, Nov 27, 2016 at 9:46 PM, Michał Jastrzębski <inc007 at gmail.com>
> wrote:
>>
>> Hey,
>>
>> I am also working with Snap community to enable log forwarding with it
>> [1].
>> Snap is super lightweight and additional benefit of this solution
>> would be that it can also handle monitoring, which was it's initial
>> role. One service to handle both would be elegant. I'll keep you
>> posted but let's not throw away this idea just yet.
>>
>> Cheers,
>> Michal
>>
>> [1] https://github.com/intelsdi-x/snap
>>
>> On 26 November 2016 at 23:55, Jeffrey Zhang <zhang.lei.fly at gmail.com>
>> wrote:
>> > Heka is marked deprecated in Kolla during Newton cycle[0]. And Now we
>> > have a
>> > blueprint for this[1]. Two alternatives, fluentd[3] and Filebeat.
>> >
>> > For Filebeat, it is just a replacement of logstash-forward[2]. It is not
>> > intent
>> > to replace the Logstash at all.
>> >
>> >> Filebeat is based on the Logstash Forwarder source code and replaces
>> >> Logstash
>> >> Forwarder as the method to use for tailing log files and forwarding
>> >> them
>> >> to
>> >> Logstash.
>> >
>> > Fillebeat is a log transport tool rather than log processing too. I do
>> > not
>> > treat it as an alternative at all.
>> >
>> > To be honest, I'd like back to Logstash, and Logstash 5.x is released
>> > with
>> > high
>> > performance improvement[4].
>> >
>> >>  In our performance testing, we've seen consistent throughput increases
>> >>  across multiple configurations. In some cases, we observed up to 75%
>> >>  increase in events processed through Logstash.
>> >
>> > another benefit to using Logstash is the whole ELK stack is maintained
>> > by
>> > one
>> > community/company. It is well tested and easy to upgrade the whole stack
>> > at
>> > the
>> > same time. Using other tools may force us on certain elasticsearch
>> > release.
>> >
>> > So, I think we have to alternative tools.
>> >
>> > * Fluentd
>> > * Logstash
>> >
>> > IMO, we need to make the decision and at least prepare the migration
>> > solution now.
>> >
>> > [1] https://blueprints.launchpad.net/kolla/+spec/heka-deprecation
>> > [2]
>> >
>> > https://www.elastic.co/guide/en/beats/filebeat/current/migrating-from-logstash-forwarder.html
>> > [3] http://www.fluentd.org/
>> > [4] https://www.elastic.co/blog/logstash-5-0-0-released
>> >
>> > --
>> > Regards,
>> > Jeffrey Zhang
>> > Blog: http://xcodest.me
>> >
>> >
>> > __________________________________________________________________________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Shake Chen
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



More information about the OpenStack-dev mailing list