Thanks. Now I understand the performance metrics you guys were talking about. It'd be good if we can have some tool reporting numbers for a cloud just like 'mpstat', 'iostat' did for a system.<br><br>
<div class="gmail_quote">On Mon, Apr 9, 2012 at 3:06 PM, Tim Bell <span dir="ltr"><<a href="mailto:Tim.Bell@cern.ch">Tim.Bell@cern.ch</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-GB" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Availability metrics for me are ones that allow me to tell if the service is up, degraded or down. Each of us as we start production monitoring need to work out how many nova, glance and swift processes of which type should be running. Furthermore, we need to add basic ‘ping’ style probes to see that the services are responding as expected. <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Performance metrics are for cases where we want to record how well the system is running. Examples of number of REST calls/second, VMs created/second etc. These are the kind of metrics which feed into capacity planning, bottleneck identification, trending.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Building up an open, standard and consistent set will avoid duplicate effort as sites deploy to production and allow us to keep the monitoring up to date when the internals of OpenStack change.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Tim<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Huang Zhiteng [mailto:<a href="mailto:winston.d@gmail.com" target="_blank">winston.d@gmail.com</a>] <br>
<b>Sent:</b> 09 April 2012 05:42<br><b>To:</b> Tim Bell<br><b>Cc:</b> David Kranz; Andrew Clay Shafer; <a href="mailto:openstack-operators@lists.openstack.org" target="_blank">openstack-operators@lists.openstack.org</a>; Duncan McGreggor; openstack</span></p>
<div><div class="h5"><br><b>Subject:</b> Re: [Openstack] [Ops] OpenStack and Operations: Input from the Wild<u></u><u></u></div></div><p></p></div></div><div><div class="h5"><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal" style="margin-bottom:12.0pt">
Hi Tim,<br><br>Could you elaborate more on 'performance metrics'? Like what kind of metrics are considered as performance ones? Thanks.<u></u><u></u></p><div><p class="MsoNormal">On Sat, Apr 7, 2012 at 2:13 AM, Tim Bell <<a href="mailto:Tim.Bell@cern.ch" target="_blank">Tim.Bell@cern.ch</a>> wrote:<u></u><u></u></p>
<div><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Splitting monitoring into</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">1.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Gathering of metrics (availability, performance) and reporting in a standard fashion should be part of OpenStack. </span><u></u><u></u></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">2.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Best practice sensors should sample the metrics and provide alarms for issues which could cause service impacts. Posting of these alarms to a monitoring system should be based on plug ins</span><u></u><u></u></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">3.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Reference implementations for standard monitoring systems such as Nagios should be available that queries the data above and feeds it into the package selected</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Each site does not want to be involved in defining the best practice. Equally, each monitoring system should not have to have an intimate understanding of OpenStack to produce a red/green light. The components for 1 and 2 fall under the associated openstack component. Component 3 is the monitoring solution provider.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Tim</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> openstack-bounces+tim.bell=<a href="mailto:cern.ch@lists.launchpad.net" target="_blank">cern.ch@lists.launchpad.net</a> [mailto:<a href="mailto:openstack-bounces%2Btim.bell" target="_blank">openstack-bounces+tim.bell</a>=<a href="mailto:cern.ch@lists.launchpad.net" target="_blank">cern.ch@lists.launchpad.net</a>] <b>On Behalf Of </b>David Kranz<br>
<b>Sent:</b> 06 April 2012 16:44<br><b>To:</b> Andrew Clay Shafer<br><b>Cc:</b> <a href="mailto:openstack-operators@lists.openstack.org" target="_blank">openstack-operators@lists.openstack.org</a>; openstack; Duncan McGreggor<br>
<b>Subject:</b> Re: [Openstack] [Ops] OpenStack and Operations: Input from the Wild</span><u></u><u></u></p></div></div><div><div><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">This is a really great list! With regard to cluster health and monitoring, I did a bunch of stuff with Swift before turning to nova and really appreciated the <br>
way each swift service has a "healthcheck" call that can be used by a monitoring system. While I don't think providing a production-ready monitoring system should be part of core OpenStack, it is the core architects who really know what needs to be checked to ensure that a system is healthy. There are various sets of poking at ports, process lists and so on that Crowbar, Zenoss, etc. set up but it would be a big improvement for deployers if each openstack service provided healthcheck apis based on expert knowledge of what is supposed to be happening inside. That would also insulate deployers from changes in the code that might impact what it means to be running properly. Looking forward to the discussion.<br>
<br> -David<br><br><br><br>On 4/6/2012 1:06 AM, Andrew Clay Shafer wrote: <u></u><u></u></p><div><p class="MsoNormal">Interested in devops.<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">
Off the top of my head.<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">live upgrades<u></u><u></u></p></div><div><p class="MsoNormal">api queryable indications of cluster health<u></u><u></u></p>
</div><div><p class="MsoNormal">api queryable cluster version and configuration info<u></u><u></u></p></div><div><p class="MsoNormal">enabling monitoring as a first class concern in OpenStack (either as a cross cutting concern, or as it's own project)<u></u><u></u></p>
</div><div><p class="MsoNormal">a framework for gathering and sharing performance benchmarks with architecture and configuration<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><p class="MsoNormal">
<u></u><u></u></p><div><p class="MsoNormal">On Thu, Apr 5, 2012 at 1:52 PM, Duncan McGreggor <<a href="mailto:duncan@dreamhost.com" target="_blank">duncan@dreamhost.com</a>> wrote:<u></u><u></u></p><p class="MsoNormal">
For anyone interested in DevOps, Ops, cloud hosting management, etc.,<br>there's a proposed session we could use your feedback on for topics of<br>discussion:<br> <a href="http://summit.openstack.org/sessions/view/57" target="_blank">http://summit.openstack.org/sessions/view/57</a><br>
<br>Respond with your thoughts and ideas, and I'll be sure to add them to the list.<br><br>Thanks!<br><br>d<br><br>_______________________________________________<br>Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><u></u><u></u></p></div><p class="MsoNormal" style="margin-bottom:12.0pt"><br><br><br><u></u><u></u></p><pre>
_______________________________________________<u></u><u></u></pre><pre>Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><u></u><u></u></pre><pre>Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><u></u><u></u></pre>
<pre>Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><u></u><u></u></pre><pre>More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><u></u><u></u></pre>
<p class="MsoNormal"> <u></u><u></u></p></div></div></div></div></div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>_______________________________________________<br>Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><u></u><u></u></p></div><p class="MsoNormal"><br><br clear="all"><br>-- <br>Regards<br>Huang Zhiteng<u></u><u></u></p>
</div></div></div></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Regards<br>Huang Zhiteng<br>