Open Stack

Hi Doug,


The basic process as I see it would be that Openstack ships with the two logging modules by default (stderr and syslog) however adding a new logging mechanism (to sentry, graphite or whatever) would be as easy as dropping a new file into nova/openstack/common/log_engines/ and enabling it in nova.conf



If I have understood your comments below, this functionality already exists via Oslo?



What we would like to see is the ability to configure multiple log destinations that can be used at the same time.  Often enterprises require the use of Syslog for compliance reasons, however it would be nice to be able to meet this requirement *and* enable developers to send "copies" of certain log events to other destinations if they need to.



Hadoop was just an example that there may be other things in future that we require that we haven't thought of yet.  At that point, it would be far easier to drop some code into a directory and enable it in nova.conf (or where ever appropriate) than rewrite the logging section of the code.



Kind regards,



Matt

From: Doug Hellmann [mailto:doug.hellmann at dreamhost.com]
Sent: 26 June 2013 14:04
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [nova] Making logging more modular



On Wed, Jun 26, 2013 at 6:31 AM, Macdonald-Wallace, Matthew <matthew.macdonald-wallace at hp.com<mailto:matthew.macdonald-wallace at hp.com>> wrote:
Hi all,

One of the things we'd like to start thinking about is having the ability to log to multiple destinations at the same time.

This would include a move towards making the logging section of Nova more like the API and Network sections in that the end user would be able to include multiple logging "engines" simply by specifying them in the log file.

Our current use case is that we want to be able to log to SYSLOG at one level (DEBUG) and have errors reported to Sentry [0] at another (WARN|ERROR) without writing complex code that analyses syslog and then pushes to sentry.

In future, we may want to send all our logs that are WARN to one location (Hadoop or similar?), ERR to another location (Monitoring?) and store the last "n" hours of debug on local disk or some other crazy configuration - moving to modular logging would enable us to do all these things.

I've not created a blueprint yet for this because I thought it best to test the water first - is this something that others in the community would be interested in seeing/working on?

Python's logging module supports this sort of thing by specifying different log handlers for different named loggers and log levels. The Oslo log module exposes these configuration settings through the --log-config option, so it is possible for a deployer to use Python's built-in file configuration to completely override the way any OpenStack component logs.

Do you foresee needing to do any work beyond creating hadoop and/or sentry log handlers? Am I misunderstanding what you  intend about including logging "engines" in the configuration file?

Doug



Kind regards,

Matt

[0] http://www.getsentry.com/

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130626/17e14db7/attachment.html>