Open Stack

Il 07/10/2014 16:31, Jeremy Stanley ha scritto:
> Indeed, OpenStack's project infrastructure and quality assurance
> teams have been collaboratively managing a very large
> logstash+elasticsearch cluster for use in classifying bugs witnessed
> while performing CI testing on proposed changes. The initial lack of
> consistency between the log formats of various services was
> maddening, and so we've been helping drive increased convergence
> over subsequent releases (hopefully you'll be pleased with the
> improvements there in Juno!). For the moment, our grok rules are
> here if it helps anyone:
> 
> <URL: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/logstash/indexer.conf.erb>


We are using this python handler:
https://github.com/bigfootproject/python-logstash

to send logging objects directly to logstash instead of passing through
the text/syslog format and then having to parse it. This way we have
richer data already available and exceptions, for example, are formatted
and do not get cut by maximum syslog message length limits.

The problem is that to use a custom handler you need to use the
log_config_append option that disables completely any other kind of log
config, so we lost the useful per-process logfile in /var/log (it cannot
be replicated by an external logging config).

There is a json log handler in oslo.log, but we could not understand how
to use/activate it. Is someone using that without the log_config_append
option?

Best,
Daniele