Open Stack

On 11/06/2017 08:36 AM, Juan Antonio Osorio wrote:
> Giving this a bit more thought; I'm slightly more inclined on merely adding
> the JSON formatter option to the standard oslo.log configuration options.

Fine for me.

> 
> This is because we right now have the ability to pass oslo.cfg options via
> the CLI, and we would loose that with the advanced logging configuration
> file. The aforementioned option is something that we're using for
> containerized openstack services.

OK - I'll check on my own if I can get something nice with the
logging.conf file. But that won't be for "tomorrow", as I'm not
full-time on openstack (unfortunately :(). Both solutions have their
pros and cons in the end.

> 
> I'll look into adding the ability to turn that handler on/off from oslo.log.

Ping me if I can help :). And big thanks for that coming change!

> 
> 
> 
> On Mon, Nov 6, 2017 at 8:34 AM, Juan Antonio Osorio <jaosorior at gmail.com>
> wrote:
> 
>>
>>
>> On Mon, Nov 6, 2017 at 8:04 AM, Cédric Jeanneret <
>> cedric.jeanneret at camptocamp.com> wrote:
>>
>>> On 11/04/2017 07:08 AM, Doug Hellmann wrote:
>>>> Excerpts from Juan Antonio Osorio's message of 2017-11-04 00:11:47
>>> +0200:
>>>>>> On 3 Nov 2017 19:59, "Doug Hellmann" <doug at doughellmann.com> wrote:
>>>>>>
>>>>>> Excerpts from Cédric Jeanneret's message of 2017-11-01 14:54:34 +0100:
>>>>>>> Dear Stackers,
>>>>>>>
>>>>>>> While working on my locally deployed Openstack (Pike using TripleO),
>>> I
>>>>>>> was a bit struggling with the logging part. Currently, all logs are
>>>>>>> pushed to per-service files, in the standard format "one line per
>>>>>>> entry", plain flat text.
>>>>>>>
>>>>>>> It's nice, but if one is wanting to push and index those logs in an
>>> ELK,
>>>>>>> the current, default format isn't really good.
>>>>>>>
>>>>>>> After some discussions about oslo.log, it appears it provides a nice
>>>>>>> JSONFormatter handler¹ one might want to use for each (python)
>>> service
>>>>>>> using oslo central library.
>>>>>>>
>>>>>>> A JSON format is really cool, as it's easy to parse for machines,
>>> and it
>>>>>>> can be on a multi-line without any bit issue - this is especially
>>>>>>> important for stack traces, as their output is multi-line without
>>> real
>>>>>>> way to have a common delimiter so that we can re-format it and feed
>>> it
>>>>>>> to any log parser (logstash, fluentd, …).
>>>>>>>
>>>>>>> After some more talks, olso.log will not provide a unified interface
>>> in
>>>>>>> order to output all received logs as JSON - this makes sens, as that
>>>>>>> would mean "rewrite almost all the python logging management
>>>>>>> interface"², and that's pretty useless, since (all?) services have
>>> their
>>>>>>> own "logging.conf" file.
>>>>>>>
>>>>>>> That said… to the main purpose of that mail:
>>>>>>>
>>>>>>> - Default format for logs
>>>>>>> A first question would be "are we all OK with the default output
>>> format"
>>>>>>> - I'm pretty sure "humans" are happy with that, as it's really
>>>>>>> convenient to read and grep. But on a "standard" Openstack deploy,
>>> I'm
>>>>>>> pretty sure one does not have only one controller, one ceph node and
>>> one
>>>>>>> compute. Hence comes the log centralization, and with that, the log
>>>>>>> indexation and treatments.
>>>>>>>
>>>>>>> For that, one might argue "I'm using plain files on my logger, and
>>>>>>> grep-ing -r in them". That's a way to do things, and for that, plain,
>>>>>>> flat logs are great.
>>>>>>>
>>>>>>> But… I'm pretty sure I'm not the only one wanting to use some kind of
>>>>>>> ELK cluster for that kind of purpose. So the right question is: what
>>>>>>> about switching the default log format to JSON? On my part, I don't
>>> see
>>>>>>> "cons", only "pros", but my judgment is of course biased, as I'm
>>> "alone
>>>>>>> in my corner". But what about you, Community?
>>>>>>>
>>>>>>> - Provide a way to configure the output format/handler
>>>>>>> While poking around in the puppet modules code, I didn't find any
>>> way to
>>>>>>> set the output handler for the logs. For example, in puppet-nova³ we
>>> can
>>>>>>> set a lot of things, but not the useful handler for the output.
>>>>>>>
>>>>>>> It would be really cool to get, for each puppet module, the
>>> capability
>>>>>>> to set the handler so that one can just push some stuff in hiera, and
>>>>>>> Voilà, we have JSON logs.
>>>>>>>
>>>>>>> Doing so would allow people to chose between the default  (current)
>>>>>>> output, and something more "computable".
>>>>>>
>>>>>> Using the JSON formatter currently requires setting a logging
>>>>>> configuration file using the standard library configuration format
>>>>>> and fully specifying things like log levels, handlers, and output
>>>>>> destination. Would it make sense to add an option in oslo.log to
>>>>>> give deployers an easier way to enable the JSON formatter?
>>>>>>
>>>>>
>>>>> This would actually be very useful.
>>>>
>>>> Great! Let me know if you would like to help figuring out where to do
>>>> that in the library code.
>>>
>>> There's already some (not really documented) feature allowing oslo.log
>>> to load a configuration file. In fact, there's already one existing in
>>> the keystone tree (/etc/keystone/logging.conf - altougn not loaded) - we
>>> might base something "casual" and "generic" on that one.
>>>
>>> I think all wsgi/python services should configure the logging through
>>> that file so that it's clearer and cleaner. But that part is maybe a bit
>>> too big and "aggressive" :). And the logging configuration isn't that
>>> friendly, to be honest, at least I have some issues with its doc ;).
>>>
>>> But I think we might come to something nice and cool. It would allow
>>> anyone to push  their own log "formatter", in the end.
>>> So you (oslo.log) wouldn't need to work with format output requests
>>> anymore, just provide two basics (plain and json), and let users play
>>> with the logging configuration (and even output things in XML if they
>>> want) ;).
>>>
>>> Regarding oslo.log: apparently, adding the following entry in the
>>> service configuration file should do it:
>>> log-config-append¹
>>>
>>> Can anyone confirm that?
>>>
>>
>> It seems to be the case at least from the docs in the option [1]. But if
>> we use this file (in TripleO and puppet) we really need to make it
>> backwards compatible. Would folks be OK with taking it into use? I guess
>> what it would take would be to document better the usage of this advanced
>> configuration. Taking it into use doesn't look that bad; If you look at the
>> file where the options are being set (_options.py), you will see that there
>> are several options that get ignored once we start using this file [2]. To
>> see all the options that get ignored you can look at the instances of
>> _IGNORE_MESSAGE. So, if we would start taking that into use, we would need
>> to change the parameters of the oslo::log resource in puppet [3] to also
>> configure an equivalent option to that advanced logging file.
>>
>> [1] https://github.com/openstack/oslo.log/blob/master/oslo_log/_
>> options.py#L44
>> [2] https://github.com/openstack/oslo.log/blob/master/oslo_log/_
>> options.py#L32
>> [3] https://github.com/openstack/puppet-oslo/blob/master/manifests/log.pp
>>
>>>
>>>
>>> ¹ https://github.com/openstack/oslo.log/blob/master/oslo_log/_
>>> options.py#L44
>>>
>>>>
>>>> Doug
>>>>
>>>> ____________________________________________________________
>>> ______________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe: OpenStack-dev-request at lists.op
>>> enstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
>>>
>>>
>>> ____________________________________________________________
>>> ______________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>>> e
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Juan Antonio Osorio R.
>> e-mail: jaosorior at gmail.com
>>
>>
> 
> 
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 878 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171106/3d995af2/attachment.sig>