<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jun 26, 2013 at 9:18 AM, Macdonald-Wallace, Matthew <span dir="ltr"><<a href="mailto:matthew.macdonald-wallace@hp.com" target="_blank">matthew.macdonald-wallace@hp.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Hi Doug,<u></u><u></u></span></p>
<p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p>The basic process as I see it would be that Openstack ships with the two logging modules by default (stderr and syslog) however adding a new logging mechanism (to sentry, graphite or whatever) would be as easy as dropping a new file
into nova/openstack/common/log_engines/ and enabling it in nova.conf<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>If I have understood your comments below, this functionality already exists via Oslo?</p></div></div></blockquote><div><br></div><div style>Right. Oslo lets you provide a logging configuration file, and the Python standard library logging module reads that file and uses is to set up different log handlers and formatters. <a href="http://docs.python.org/2/library/logging.html">http://docs.python.org/2/library/logging.html</a></div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div>
<p><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>What we would like to see is the ability to configure multiple log destinations that can be used at the same time. Often enterprises require the use of Syslog for compliance reasons, however it would be nice to be able to meet this
requirement *<b>and</b>* enable developers to send “copies” of certain log events to other destinations if they need to.</p></div></div></blockquote><div><br></div><div style>Yes, this is definitely possible with the existing library. I think you'll just need to write the bits that plug in, and then document how to configure them in the logging config file. No changes should be needed in Oslo or Nova.<br>
</div><div style><br>Doug</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple">
<div><p><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Hadoop was just an example that there may be other things in future that we require that we haven’t thought of yet. At that point, it would be far easier to drop some code into a directory and enable it in nova.conf (or where ever appropriate)
than rewrite the logging section of the code. </p></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><div><p><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Kind regards,<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Matt<u></u><u></u></p>
<p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<div style="border-style:none none none solid;border-left-color:blue;border-left-width:1.5pt;padding:0cm 0cm 0cm 4pt">
<div>
<div style="border-style:solid none none;border-top-color:rgb(181,196,223);border-top-width:1pt;padding:3pt 0cm 0cm">
<p class=""><b><span style="font-size:10pt;font-family:Tahoma,sans-serif">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif"> Doug Hellmann [mailto:<a href="mailto:doug.hellmann@dreamhost.com" target="_blank">doug.hellmann@dreamhost.com</a>]
<br>
<b>Sent:</b> 26 June 2013 14:04<br>
<b>To:</b> OpenStack Development Mailing List<br>
<b>Subject:</b> Re: [openstack-dev] [nova] Making logging more modular<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class=""><u></u> <u></u></p>
<div>
<p class=""><u></u> <u></u></p>
<div>
<p class="" style="margin-bottom:12pt"><u></u> <u></u></p>
<div>
<p class="">On Wed, Jun 26, 2013 at 6:31 AM, Macdonald-Wallace, Matthew <<a href="mailto:matthew.macdonald-wallace@hp.com" target="_blank">matthew.macdonald-wallace@hp.com</a>> wrote:<u></u><u></u></p>
<p class="">Hi all,<br>
<br>
One of the things we'd like to start thinking about is having the ability to log to multiple destinations at the same time.<br>
<br>
This would include a move towards making the logging section of Nova more like the API and Network sections in that the end user would be able to include multiple logging "engines" simply by specifying them in the log file.<br>
<br>
Our current use case is that we want to be able to log to SYSLOG at one level (DEBUG) and have errors reported to Sentry [0] at another (WARN|ERROR) without writing complex code that analyses syslog and then pushes to sentry.<br>
<br>
In future, we may want to send all our logs that are WARN to one location (Hadoop or similar?), ERR to another location (Monitoring?) and store the last "n" hours of debug on local disk or some other crazy configuration - moving to modular logging would enable
us to do all these things.<br>
<br>
I've not created a blueprint yet for this because I thought it best to test the water first - is this something that others in the community would be interested in seeing/working on?<u></u><u></u></p>
<div>
<p class=""><u></u> <u></u></p>
</div>
<div>
<p class="">Python's logging module supports this sort of thing by specifying different log handlers for different named loggers and log levels. The Oslo log module exposes these configuration settings through the --log-config option, so it is possible
for a deployer to use Python's built-in file configuration to completely override the way any OpenStack component logs.<u></u><u></u></p>
</div>
<div>
<p class=""><u></u> <u></u></p>
</div>
<div>
<p class="">Do you foresee needing to do any work beyond creating hadoop and/or sentry log handlers? Am I misunderstanding what you intend about including logging "engines" in the configuration file?<u></u><u></u></p>
</div>
<div>
<p class=""><u></u> <u></u></p>
</div>
<div>
<p class="">Doug<u></u><u></u></p>
</div>
<div>
<p class=""><u></u> <u></u></p>
</div>
<div>
<p class=""> <u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-left-color:rgb(204,204,204);border-left-width:1pt;padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<p class=""><br>
Kind regards,<br>
<br>
Matt<br>
<br>
[0] <a href="http://www.getsentry.com/" target="_blank">http://www.getsentry.com/</a><br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><u></u><u></u></p>
</blockquote>
</div>
<p class=""><u></u> <u></u></p>
</div>
</div>
</div></div></div>
</div>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>