Open Stack

sbezverk,

yep. Event though I prefer to use Logstash. But fluentd is OK to me.

On Mon, Nov 28, 2016 at 8:31 AM, Serguei Bezverkhi (sbezverk) <
sbezverk at cisco.com> wrote:

> I would vote for fluentd because: 1 – it has been around since 2011 so it
> is hard to call it green. 2 – There is constant development of new
> features/filters, 3 – As it was mentioned at Kubecon by fluentd people,
> they are deeply committed to Open Source community so rumors that they
> would go private does not sound reliable. We use it in kolla kubernetes and
> so far we could share only positive feedback.
>
>
>
> Serguei
>
>
>
> *From:* Steven Dake (stdake)
> *Sent:* Sunday, November 27, 2016 3:46 PM
>
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev at lists.openstack.org>
> *Subject:* Re: [openstack-dev] [kolla] the alternative of log processing
> tool
>
>
>
> Jeffrey,
>
>
>
> Logstash-forwarder is deprecated upstream, so we can’t rely on that.
> Elastic's replacement is filebeat.
>
>
>
> I’m not sure which one meets the requirements – filebeat or fluentd.  In
> kolla-kubernetes fluentd is being used, and is well maintained.  Both
> implementations are pretty green IMO.  Not sure if fluentd also does log
> processing.  I think its crucial to pick a component that just does log
> forwarding since that is the part that was deprecated.
>
>
>
> Our system has no log stash at all in it, and I’d like to keep it that
> way.  Logstash is unnecessary for our use case.  What we want is
> forwarder->es->cabana.  Whatever forwarder is chosen, recommend picking the
> best of the two choices.  I’d start with defining best as “does it solve
> the same problem as Heka does in our current implementation” then sprinkle
> throughput and minimal cpu and network utilization on top.  If we can’t
> make a decision from there, not sure I have any further suggestions as I am
> not writing the code.
>
>
>
> Regards
>
> -steve
>
>
>
>
>
> *From: *Jeffrey Zhang <zhang.lei.fly at gmail.com>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev at lists.openstack.org>
> *Date: *Sunday, November 27, 2016 at 9:40 AM
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> *Subject: *Re: [openstack-dev] [kolla] the alternative of log processing
> tool
>
>
>
> So filebeat is working with Logstash right? We need split the logs into
> pieces by using logstash. IMU, Filebeat do not a variety of processing
> plugins, like Logstash[0].
>
>
>
> [0] https://www.elastic.co/guide/en/logstash/current/filter-plugins.html
>
>
>
> On Sun, Nov 27, 2016 at 11:30 PM, Ian Cordasco <sigmavirus24 at gmail.com>
> wrote:
>
> File beat is maintained be elastic and a part of their product line just
> like ELK. It's a fantastic tool and quite flexible given its age and size
> of codebase
>
>
>
> On Nov 26, 2016 11:59 PM, "Jeffrey Zhang" <zhang.lei.fly at gmail.com> wrote:
>
> Heka is marked deprecated in Kolla during Newton cycle[0]. And Now we have
> a
>
> blueprint for this[1]. Two alternatives, fluentd[3] and Filebeat.
>
>
>
> For Filebeat, it is just a replacement of logstash-forward[2]. It is not
> intent
>
> to replace the Logstash at all.
>
>
>
> > Filebeat is based on the Logstash Forwarder source code and replaces
> Logstash
>
> > Forwarder as the method to use for tailing log files and forwarding them
> to
>
> > Logstash.
>
>
>
> Fillebeat is a log transport tool rather than log processing too. I do not
>
> treat it as an alternative at all.
>
>
>
> To be honest, I'd like back to Logstash, and Logstash 5.x is released with
> high
>
> performance improvement[4].
>
>
>
> >  In our performance testing, we've seen consistent throughput increases
>
> >  across multiple configurations. In some cases, we observed up to 75%
>
> >  increase in events processed through Logstash.
>
>
>
> another benefit to using Logstash is the whole ELK stack is maintained by
> one
>
> community/company. It is well tested and easy to upgrade the whole stack
> at the
>
> same time. Using other tools may force us on certain elasticsearch release.
>
>
>
> So, I think we have to alternative tools.
>
>
>
> * Fluentd
>
> * Logstash
>
>
>
> IMO, we need to make the decision and at least prepare the migration
> solution now.
>
>
>
> [1] https://blueprints.launchpad.net/kolla/+spec/heka-deprecation
>
> [2] https://www.elastic.co/guide/en/beats/filebeat/current/
> migrating-from-logstash-forwarder.html
>
> [3] http://www.fluentd.org/
>
> [4] https://www.elastic.co/blog/logstash-5-0-0-released
>
>
>
> --
>
> Regards,
>
> Jeffrey Zhang
>
> Blog: http://xcodest.me
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>
> --
>
> Regards,
>
> Jeffrey Zhang
>
> Blog: http://xcodest.me
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Regards,
Jeffrey Zhang
Blog: http://xcodest.me
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161128/360270cb/attachment.html>