<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>I have a solution. At least it works for me. Be aware that this
is Devstack, but I think nothing I did to solve my problem is
Devstack-specific. Also, I don't know whether there are more
efficient or canonical ways to reconfigure Ceilometer. But it's
good enough for me.<br>
</p>
<p><br>
</p>
These are my steps - you may not need all of them.<br>
<ul>
<li>in <b>pipeline.yaml</b>, set publisher to gnocchi://</li>
<li>in <b>the resource definition file</b>, define my new archive
policy.<br>
By default, this file resides in the Ceilometer source tree
.../ceilometer/publisher/data/gnocchi_resources.yaml, but you
can use config parameter <span class="c1">resources_definition_file
to change the default (I didn't try).<br>
Example:</span><span class="c1"></span></li>
</ul>
<p><tt> - name: ceilometer-medium-rate</tt><tt><br>
</tt><tt> aggregation_methods:</tt><tt><br>
</tt><tt> - mean</tt><tt><br>
</tt><tt> - rate:mean</tt><tt><br>
</tt><tt> back_window: 0</tt><tt><br>
</tt><tt> definition:</tt><tt><br>
</tt><tt> - granularity: 1 minute</tt><tt><br>
</tt><tt> timespan: 7 days</tt><tt><br>
</tt><tt> - granularity: 1 hour</tt><tt><br>
</tt><tt> timespan: 365 days</tt><br>
</p>
<ul>
<li>in the same resource definition file, <b>adjust the archive
policy </b>of rate metrics.<br>
Example:</li>
</ul>
<tt> - resource_type: instance</tt><tt><br>
</tt><tt> metrics:</tt><tt><br>
</tt><tt> ...</tt><tt><br>
</tt><tt> cpu:</tt><tt><br>
</tt><tt> archive_policy_name: ceilometer-medium-rate</tt><tt><br>
</tt>
<ul>
<li><b>delete all existing metrics and resources </b>from Gnocchi<br>
Probably only necessary when Ceilometer is running, and not
needed if you reconfigure it before its first start.<br>
This is a drastic measure, but if you do it at the beginning of
a deployment, it won't cause loss of much data. <br>
Why is this required? A metric contains an archive policy that
can't be changed. Thus existing metrics need to be recreated.<br>
Why remove resources? Because they reference the metrics that I
removed.<br>
</li>
</ul>
<ul>
<li><b>restart all Ceilometer services</b><br>
This is required for re-reading the pipeline and the resource
definition files.<br>
Ceilometer will create resources and metrics as needed when it
sends its samples to Gnocchi.</li>
</ul>
<p>I tested this by running a CPU hogging instance and listing its
measures after a few minutes:</p>
<p><br>
</p>
<p><tt> gnocchi measures show --resource
f28f6b78-9dd5-49cc-a6ac-28cb14477bf0 </tt><tt><br>
</tt><tt> --aggregation rate:mean cpu</tt></p>
<p><tt>
+---------------------------+-------------+---------------+<br>
| timestamp | granularity | value |<br>
+---------------------------+-------------+---------------+<br>
| 2019-08-01T20:23:00+09:00 | 60.0 | 1810000000.0 |<br>
| 2019-08-01T20:24:00+09:00 | 60.0 | 39940000000.0 |<br>
| 2019-08-01T20:25:00+09:00 | 60.0 | 40110000000.0 |<br>
</tt></p>
<p><br>
</p>
<p>This means that the instance accumulated 39940000000 nanoseconds
of CPU time in the 60 seconds at <br>
20:24:00. Note that the old <i>cpu_util </i>was expressed in
percent, so that Aodh alarms and Heat autoscaling definitions must
be adapted.<br>
</p>
<p><br>
</p>
<p>Good luck. Hire me as Ceilometer consultant if you get stuck :)
<br>
</p>
<p><br>
</p>
<p>Bernd</p>
<p><br>
</p>
On 8/1/2019 6:11 PM, Teckelmann, Ralf, NMU-OIP wrote:<br>
<blockquote type="cite"
cite="mid:AM6PR04MB47117DC1D88A74F0242C7A0E89DE0@AM6PR04MB4711.eurprd04.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper"
style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;"
dir="ltr">
<p style="margin-top:0;margin-bottom:0">Hello Bernd, Hello <span>Lingxian</span>,</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">+1</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">You are not alone in
your fruitless endeavor. Sadly, I can not come up with a
solution.</p>
<p style="margin-top:0;margin-bottom:0">We are stuck at the same
point.<br>
</p>
<span style="font-size:10pt;
font-family:"Arial",sans-serif,serif,"EmojiFont""></span>
<div><br>
</div>
<div>Maybe some day a dedicated member of the OpenStack
community give the ceilometer guys a push to explain their
service.
<br>
</div>
<div>For us, also using Stein, it is in the state of "not
production ready".<br>
</div>
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Ralf T.<br>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt"
face="Calibri, sans-serif" color="#000000"><b>Von:</b> Bernd
Bausch <a class="moz-txt-link-rfc2396E" href="mailto:berndbausch@gmail.com"><berndbausch@gmail.com></a><br>
<b>Gesendet:</b> Donnerstag, 1. August 2019 03:16:25<br>
<b>An:</b> Lingxian Kong <a class="moz-txt-link-rfc2396E" href="mailto:anlin.kong@gmail.com"><anlin.kong@gmail.com></a><br>
<b>Cc:</b> openstack-discuss
<a class="moz-txt-link-rfc2396E" href="mailto:openstack-discuss@lists.openstack.org"><openstack-discuss@lists.openstack.org></a><br>
<b>Betreff:</b> Re: [telemetry][ceilometer][gnocchi] How to
configure aggregate for cpu_util or calculate from metrics</font>
<div> </div>
</div>
<div style="background-color:#FFFFFF">
<p>Lingxian, <br>
</p>
<p>Thanks for "bumping" my request and keeping it alive. The
reason I need an answer: I am updating courseware to Stein
that includes autoscaling based on CPU and disk I/O rates.
Looks like I am "cutting edge" :)<br>
</p>
<p>I don't think the problem is in the Gnocchi camp, but rather
Ceilometer. To store rates of measures in z, the following is
needed:</p>
<ul>
<li>A <i>metric</i>. Raw measures are sent to the metric.<br>
</li>
<li>An <i>archive policy</i>. The metric has an archive
policy.<br>
</li>
<li>The archive policy includes one or more <i>rate
aggregates</i></li>
</ul>
My cloud has archive policies with rate aggregates, but the
question is about the first bullet:
<b>How can I configure Ceilometer so that it creates the
corresponding metrics and sends measures to them.
</b>In other words, how is Ceilometer's output connected to my
archive policy. From my experience, just adding the archive
policy to Ceilometer's publishers is not sufficient.
<br>
<p>Ceilometer's source code includes <i>.../publisher/data/gnocchi_resources.yaml</i>,
which might well be the place where this can be configured. I
am not sure how to do it though, and this file is not
documented. I can read the source, but my developer skills are
insufficient for understanding how everything fits together.<br>
</p>
<p>Bernd<br>
</p>
<div class="x_moz-cite-prefix">On 8/1/2019 9:01 AM, Lingxian
Kong wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="x_gmail_default"
style="font-family:arial,helvetica,sans-serif">Hi Bernd,</div>
<div class="x_gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
</div>
<div class="x_gmail_default"
style="font-family:arial,helvetica,sans-serif">There were
a lot of people asked the same question before,
unfortunately, I don't know the answer either(we are still
using an old version of Ceilometer). The original cpu_util
support has been removed from Ceilometer in favor of
Gnocchi, but AFAIK, there is no doc in Gnocchi mentioned
how to achieve the same thing and no clear answer from the
Gnocchi maintainers.</div>
<div class="x_gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
</div>
<div class="x_gmail_default"
style="font-family:arial,helvetica,sans-serif">It'd be
much appreciated if you could find the answer in the end,
or there will be someone who has the already solved the
issue.</div>
<div>
<div dir="ltr" class="x_gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div><br>
</div>
<div><font
face="monospace,
monospace"
color="#666666">Best
regards,<br>
Lingxian Kong</font></div>
<div><font
face="monospace,
monospace"
color="#666666">Catalyst
Cloud</font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Wed, Jul 31, 2019 at
1:28 PM Bernd Bausch <<a
href="mailto:berndbausch@gmail.com"
moz-do-not-send="true">berndbausch@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px
0.8ex; border-left:1px solid rgb(204,204,204);
padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>The message at the end of this email is some three
months old. I have the same problem. The question is:
<b>How to use the new rate metrics in Gnocchi. </b>I
am using a Stein Devstack for my tests.<b><br>
</b></p>
<p>For example, I need the CPU rate, formerly named <i>cpu_util</i>.
I created a new archive policy that uses
<i>rate:mean</i> aggregation and has a 1 minute
granularity:</p>
<p><tt>$ gnocchi archive-policy show
ceilometer-medium-rate</tt><tt><br>
</tt><tt>+---------------------+------------------------------------------------------------------+</tt><tt><br>
</tt><tt>| Field |
Value
|</tt><tt><br>
</tt><tt>+---------------------+------------------------------------------------------------------+</tt><tt><br>
</tt><tt>| aggregation_methods | rate:mean,
mean
|</tt><tt><br>
</tt><tt>| back_window |
0
|</tt><tt><br>
</tt><tt>| definition | - points: 10080,
granularity: 0:01:00, timespan: 7 days, 0:00:00 |</tt><tt><br>
</tt><tt>| name |
ceilometer-medium-rate
|</tt><tt><br>
</tt><tt>+---------------------+------------------------------------------------------------------+</tt><br>
</p>
<p>I added the new policy to the publishers in <i>pipeline.yaml</i>:</p>
<p><tt>$ tail -n5 /etc/ceilometer/pipeline.yaml</tt><tt><br>
</tt><tt>sinks:</tt><tt><br>
</tt><tt> - name: meter_sink</tt><tt><br>
</tt><tt> publishers:</tt><tt><br>
</tt><tt> -
gnocchi://?archive_policy=medium&filter_project=gnocchi_swift</tt><tt><br>
</tt><tt> <b>-
gnocchi://?archive_policy=ceilometer-medium-rate&filter_project=gnocchi_swift</b></tt><br>
</p>
<p>After restarting all of Ceilometer, my hope was that
the CPU rate would magically appear in the metric
list. But no: All metrics are linked to archive policy
<i>medium</i>, and looking at the details of an
instance, I don't detect anything rate-related:</p>
<p><tt>$ gnocchi resource show
ae3659d6-8998-44ae-a494-5248adbebe11</tt><tt><br>
</tt><tt>+-----------------------+---------------------------------------------------------------------+</tt><tt><br>
</tt><tt>| Field |
Value
|</tt><tt><br>
</tt><tt>+-----------------------+---------------------------------------------------------------------+</tt><tt><br>
</tt><tt>...</tt><tt><br>
</tt><tt>| metrics |
compute.instance.booting.time:
76fac1f5-962e-4ff2-8790-1f497c99c17d |</tt><tt><br>
</tt><tt>| | cpu:
af930d9a-a218-4230-b729-fee7e3796944
|</tt><tt><br>
</tt><tt>| |
disk.ephemeral.size:
0e838da3-f78f-46bf-aefb-aeddf5ff3a80 |</tt><tt><br>
</tt><tt>| | disk.root.size:
5b971bbf-e0de-4e23-ba50-a4a9bf7dfe6e
|</tt><tt><br>
</tt><tt>| | memory.resident:
09efd98d-c848-4379-ad89-f46ec526c183 |</tt><tt><br>
</tt><tt>| | <a
href="https://urldefense.proofpoint.com/v2/url?u=http-3A__memory.swap.in&d=DwMDaQ&c=vo2ie5TPcLdcgWuLVH4y8lsbGPqIayH3XbK3gK82Oco&r=WXex93lsaiQ-z7CeZkHv93lzt4fdCRIPXloSPQEU7CM&m=pnr97rQYDOFbG5UeNvvK1DDoP0YecUmqLwRt4SI4wOU&s=wDnZesKE356cMfbQrJMuwYwdEof7ULmQOFQgqE31umo&e="
target="_blank" moz-do-not-send="true">
memory.swap.in</a>:
1bb4bb3c-e40a-4810-997a-295b2fe2d5eb
|</tt><tt><br>
</tt><tt>| | memory.swap.out:
4d012697-1d89-4794-af29-61c01c925bb4 |</tt><tt><br>
</tt><tt>| | memory.usage:
93eab625-0def-4780-9310-eceff46aab7b
|</tt><tt><br>
</tt><tt>| | memory:
ea8f2152-09bd-4aac-bea5-fa8d4e72bbb1
|</tt><tt><br>
</tt><tt>| | vcpus:
e1c5acaf-1b10-4d34-98b5-3ad16de57a98
|</tt><tt><br>
</tt><tt>| original_resource_id |
ae3659d6-8998-44ae-a494-5248adbebe11
|</tt><tt><br>
</tt><tt>...</tt></p>
<p><tt>| type |
instance
|</tt><tt><br>
</tt><tt>| user_id |
a9c935f52e5540fc9befae7f91b4b3ae
|</tt><tt><br>
</tt><tt>+-----------------------+---------------------------------------------------------------------+</tt></p>
<p>Obviously, I am missing something. Where is the
missing link? What do I have to do to get CPU usage
rates? Do I have to create metrics? Do<i>
</i>I have to ask Ceilometer to create metrics? How? <br>
</p>
<p>Right now, no instructions seem to exist at all. If
that is correct, I would be happy to write
documentation once I understand how it works.
</p>
<p>Thanks a lot.</p>
<p>Bernd<br>
</p>
<div class="x_gmail-m_116874100483014509moz-cite-prefix">On
5/10/2019 3:49 PM, <a
class="x_gmail-m_116874100483014509moz-txt-link-abbreviated"
href="mailto:info@dantalion.nl" target="_blank"
moz-do-not-send="true">
info@dantalion.nl</a> wrote:<br>
</div>
<blockquote type="cite">
<pre class="x_gmail-m_116874100483014509moz-quote-pre">Hello,
I am working on Watcher and we are currently changing how metrics are
retrieved from different datasources such as Monasca or Gnocchi. Because
of this major overhaul I would like to validate that everything is
working correctly.
Almost all of the optimization strategies in Watcher require the cpu
utilization of an instance as metric but with newer versions of
Ceilometer this has become unavailable.
On IRC I received the information that Gnocchi could be used to
configure an aggregate and this aggregate would then report cpu
utilization, however, I have been unable to find documentation on how to
achieve this.
I was also notified that cpu_util is something that could be computed
from other metrics. When reading
<a class="x_gmail-m_116874100483014509moz-txt-link-freetext" href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.openstack.org_ceilometer_rocky_admin_telemetry-2Dmeasurements.html-23openstack-2Dcompute&d=DwMDaQ&c=vo2ie5TPcLdcgWuLVH4y8lsbGPqIayH3XbK3gK82Oco&r=WXex93lsaiQ-z7CeZkHv93lzt4fdCRIPXloSPQEU7CM&m=pnr97rQYDOFbG5UeNvvK1DDoP0YecUmqLwRt4SI4wOU&s=-ncji0Wl7WScsqBfumudi0ot_et_UIRfjh2c464FYWY&e=" target="_blank" moz-do-not-send="true">https://docs.openstack.org/ceilometer/rocky/admin/telemetry-measurements.html#openstack-compute</a>
the documentation seems to agree on this as it states that cpu_util is
measured by using a 'rate of change' transformer. But I have not been
able to find how this can be computed.
I was hoping someone could spare the time to provide documentation or
information on how this currently is best achieved.
Kind Regards,
Corne Lukken (Dantali0n)
</pre>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>