<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Cool ... i will go ahead and submit a blueprint for these.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Wrt the compression blueprint, locally we subclassed the base python rotating file handler class in order to optionally compress the file as part of the “shouldRollover” stage.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Greg.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-family:Calibri;color:black">From: </span>
</b><span style="font-family:Calibri;color:black">gordon chung <gord@live.ca><br>
<b>Reply-To: </b>"openstack-dev@lists.openstack.org" <openstack-dev@lists.openstack.org><br>
<b>Date: </b>Thursday, November 16, 2017 at 1:49 PM<br>
<b>To: </b>"openstack-dev@lists.openstack.org" <openstack-dev@lists.openstack.org><br>
<b>Subject: </b>Re: [openstack-dev] [ceilometer] Blueprint proposals for 'file' publisher: Compression and CSV file format<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">On 2017-11-16 12:44 PM, Waines, Greg wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal">1.Compression of files (within file rotation) to minimize disk usage and
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">minimize bandwidth in FTP-ing these files off server<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">oi.e. as part of file rotation, the newly rotated file would be gzip’d
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">to compress the file<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">othis would be an optional parameter in the pipeline definition in
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">/etc/ceilometer/pipeline.yaml<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">e.g.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">name: meter_file<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">meters:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> - "*"<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">publishers:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> - <a href="file:///var/test?max_bytes=10000000&backup_count=5&compress=true">
file:///var/test?max_bytes=10000000&backup_count=5&compress=true</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">oagain,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">minimizing disk usage and minimizing network bandwidth in FTP-ing these
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">files off server.<o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">seems like a good idea. just curious, how would you implement this?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal">2.CSV (Comma Separated Values) File Format<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">oCSV is widely supported for many off-box Statistical Analysis-type
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Tools; especially in Telecom,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">oagain this would be done as an optional argument to the file publisher<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">§the existing format (python dictionary print) would remain as the
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">default format<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">§‘format=csv’ would be the argument added to the publisher line in order
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">to enable CSV format<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">e.g.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">name: meter_file<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">meters:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> - "*"<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">publishers:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> - <a href="file:///var/test?max_bytes=10000000&backup_count=5&format=csv">
file:///var/test?max_bytes=10000000&backup_count=5&format=csv</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal">this sounds like a good idea. i believe there was a bug previously that
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">suggested cleaning up the output of the file publisher/dispatcher so
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">it's not just dumping random blurbs of json. i think the proposed
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">format=csv trigger makes sense.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal">I think these would be valuable addons to the Ceilometer File Publisher,<o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">agreed! thanks for suggesting them. this is me assuming you (or a
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">colleague) will implement this :)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">-- <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">gord<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">__________________________________________________________________________<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">OpenStack Development Mailing List (not for usage questions)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org">
OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>