[openstack-dev] [kolla] the alternative of log processing tool

Shake Chen shake.chen at gmail.com
Sun Nov 27 15:41:25 UTC 2016


Thanks for quick reply.

so I think the snap is not good choice。

For end user, hope the solustion is mature and easy maintain.

as discuss before,

I'll put on my operator hat and would like to give my +1 to keep ELK instead
of Heka.


http://lists.openstack.org/pipermail/openstack-dev/2016-January/083974.html






On Sun, Nov 27, 2016 at 11:09 PM, Michał Jastrzębski <inc007 at gmail.com>
wrote:

> It's in development
>
> On 27 November 2016 at 08:50, Shake Chen <shake.chen at gmail.com> wrote:
> > Hi Michal
> >
> > the snap seem not support log forward now, I can not find any infomation
> in
> > google .
> >
> > On Sun, Nov 27, 2016 at 9:46 PM, Michał Jastrzębski <inc007 at gmail.com>
> > wrote:
> >>
> >> Hey,
> >>
> >> I am also working with Snap community to enable log forwarding with it
> >> [1].
> >> Snap is super lightweight and additional benefit of this solution
> >> would be that it can also handle monitoring, which was it's initial
> >> role. One service to handle both would be elegant. I'll keep you
> >> posted but let's not throw away this idea just yet.
> >>
> >> Cheers,
> >> Michal
> >>
> >> [1] https://github.com/intelsdi-x/snap
> >>
> >> On 26 November 2016 at 23:55, Jeffrey Zhang <zhang.lei.fly at gmail.com>
> >> wrote:
> >> > Heka is marked deprecated in Kolla during Newton cycle[0]. And Now we
> >> > have a
> >> > blueprint for this[1]. Two alternatives, fluentd[3] and Filebeat.
> >> >
> >> > For Filebeat, it is just a replacement of logstash-forward[2]. It is
> not
> >> > intent
> >> > to replace the Logstash at all.
> >> >
> >> >> Filebeat is based on the Logstash Forwarder source code and replaces
> >> >> Logstash
> >> >> Forwarder as the method to use for tailing log files and forwarding
> >> >> them
> >> >> to
> >> >> Logstash.
> >> >
> >> > Fillebeat is a log transport tool rather than log processing too. I do
> >> > not
> >> > treat it as an alternative at all.
> >> >
> >> > To be honest, I'd like back to Logstash, and Logstash 5.x is released
> >> > with
> >> > high
> >> > performance improvement[4].
> >> >
> >> >>  In our performance testing, we've seen consistent throughput
> increases
> >> >>  across multiple configurations. In some cases, we observed up to 75%
> >> >>  increase in events processed through Logstash.
> >> >
> >> > another benefit to using Logstash is the whole ELK stack is maintained
> >> > by
> >> > one
> >> > community/company. It is well tested and easy to upgrade the whole
> stack
> >> > at
> >> > the
> >> > same time. Using other tools may force us on certain elasticsearch
> >> > release.
> >> >
> >> > So, I think we have to alternative tools.
> >> >
> >> > * Fluentd
> >> > * Logstash
> >> >
> >> > IMO, we need to make the decision and at least prepare the migration
> >> > solution now.
> >> >
> >> > [1] https://blueprints.launchpad.net/kolla/+spec/heka-deprecation
> >> > [2]
> >> >
> >> > https://www.elastic.co/guide/en/beats/filebeat/current/
> migrating-from-logstash-forwarder.html
> >> > [3] http://www.fluentd.org/
> >> > [4] https://www.elastic.co/blog/logstash-5-0-0-released
> >> >
> >> > --
> >> > Regards,
> >> > Jeffrey Zhang
> >> > Blog: http://xcodest.me
> >> >
> >> >
> >> > ____________________________________________________________
> ______________
> >> > OpenStack Development Mailing List (not for usage questions)
> >> > Unsubscribe:
> >> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >> >
> >>
> >> ____________________________________________________________
> ______________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> > --
> > Shake Chen
> >
> >
> > ____________________________________________________________
> ______________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Shake Chen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161127/93d37229/attachment.html>


More information about the OpenStack-dev mailing list