[openstack-dev] [kolla] the alternative of log processing tool

Joshua Harlow harlowja at fastmail.com
Tue Nov 29 04:47:40 UTC 2016


Why are people/things parsing tracebacks out of log files when the 
following exists:

http://docs.openstack.org/developer/oslo.log/api/formatters.html#oslo_log.formatters.JSONFormatter

Seems like oslo.log also has a fluent formatter @ 
http://docs.openstack.org/developer/oslo.log/api/formatters.html#oslo_log.formatters.FluentFormatter.

IMHO I hope that nobody is making regex or such for tracebacks any more 
in the 21st century; structured data ftw...

Michał Jastrzębski wrote:
> Having custom /dev/log was real pain in few occasions. Also syslog was
> particularly bad in working with multi-line logging (like python
> tracebacks).
> Heka reads local log files, makes things easier, and parses things
> like tracebacks in it. It's my understanding that fluentd can do the
> same.
>
> On 28 November 2016 at 19:07, Jeffrey Zhang<zhang.lei.fly at gmail.com>  wrote:
>> On Tue, Nov 29, 2016 at 7:29 AM, Michał Jastrzębski<inc007 at gmail.com>
>> wrote:
>>> I don't really like logstash as it's big memory eating beast. We had
>>> good arch without it, and I'd like to keep it this way. Even with
>>> logstash we still would need to use rsyslog to push logs around to
>>> logstash, and that's a pita (trust me, I wrote it.).
>> About the rsyslog mock socket, i think we have fixed it.
>> heka is using rsyslog socket too. IIUC, i know the issue. could u explain
>> more?
>>>
>>> Fluentd just became cncf-backed project so that's a plus for fluentd.
>>> Out of fliebeat+logstash vs flutentd I'd go with fluentd.
>>
>> So, I am OK to use fluentd.
>>
>> ;)
>>
>>
>>
>>
>> --
>> Regards,
>> Jeffrey Zhang
>> Blog: http://xcodest.me
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list