<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Nadya,<br>
Thanks for the clarification.<br>
Do you know if there is a bug report for this issue? If not, should
I submit one for tracking?<br>
Reza<br>
<br>
<div class="moz-cite-prefix">On 4/8/2016 11:48 AM, Nadya Shakhat
wrote:<br>
</div>
<blockquote
cite="mid:CAKWrcJffKGuNeE=enSiSKxSoqjAL9XkCpdugk0eYdtkeWZaUuw@mail.gmail.com"
type="cite">
<div dir="ltr">Hello, Reza!
<div><br>
</div>
<div>Thank you very much for this question. Your understanding
is correct. In Liberty in Ceilometer we did a refactoring
which has changed the behaviour in the way you've described.
Let me try to explain in details. </div>
<div><br>
</div>
<div>Each cpu_util sample is the sample which is derived from
cpu ones. In Juno, this transformation took place in Compute
agent. In Liberty, we've started to do it in Notification
agent. And it's the root cause of the issue. Let's consider
the following pipeline.yam file:</div>
<div>
<table class=""
style="border-collapse:collapse;border-spacing:0px;color:rgb(51,51,51);font-family:Helvetica,arial,nimbussansl,liberationsans,freesans,clean,sans-serif,'Apple
Color Emoji','Segoe UI Emoji','Segoe UI
Symbol';font-size:13px;line-height:18px">
<tbody style="">
<tr style="">
<td id="LC2" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"><span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">sources:</span></span></td>
</tr>
<tr style="">
<td id="L3" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC3" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="color:rgb(99,163,92)">name:</span> <span class="" style="">meter_source</span></span></td>
</tr>
<tr style="">
<td id="L4" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC4" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(0,134,179)"><span class="" style="color:rgb(99,163,92)">interval:</span> 600</span></td>
</tr>
<tr style="">
<td id="L5" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC5" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">meters:</span></span></td>
</tr>
<tr style="">
<td id="L6" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC6" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style=""><span class="" style="">"</span>*<span class="" style="">"</span></span></span></td>
</tr>
<tr style="">
<td id="L7" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC7" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">sinks:</span></span></td>
</tr>
<tr style="">
<td id="L8" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC8" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="">meter_sink</span></span></td>
</tr>
<tr style="">
<td id="L9" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC9" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="color:rgb(99,163,92)">name:</span> <span class="" style="">cpu_source</span></span></td>
</tr>
<tr style="">
<td id="L10" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC10" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(0,134,179)"><span class="" style="color:rgb(99,163,92)">interval:</span> 60 <-----it's your change</span></td>
</tr>
<tr style="">
<td id="L11" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC11" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">meters:</span></span></td>
</tr>
<tr style="">
<td id="L12" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC12" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style=""><span class="" style="">"</span>cpu<span class="" style="">"</span></span></span></td>
</tr>
<tr style="">
<td id="L13" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC13" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">sinks:</span></span></td>
</tr>
<tr style="">
<td id="L14" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC14" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="">cpu_sink</span></span></td>
</tr>
<tr style="">
<td id="L15" class="" style="padding:0px
10px;width:50px;min-width:50px;font-family:Consolas,'Liberation
Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px
1px 0px 0px"><br>
</td>
<td id="LC15" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="">cpu_delta_sink</span></span></td>
</tr>
</tbody>
</table>
</div>
<div> </div>
<div>It's written here that all possible pollsters (see
setup.cfg <span
style="color:rgb(0,0,0);font-family:Menlo;font-size:9pt">ceilometer.poll.compute
section</span>) should be run every 10 minutes and
"meter_sink" should be applied accordingly. Also, once in 60
seconds you want to run _only_ cpu pollster (<span
style="color:rgb(0,0,0);font-family:Menlo;font-size:9pt">ceilometer.compute.pollsters.cpu:CPUPollster</span>)
and transform the result as described in cpu_sink and
cpu_delta_sink. In Juno, we did everything in Compute agent.</div>
<div> In Liberty the situation has been changed. In the
example above, we run all pollsters once in 600 seconds and
send the result to the RabbitMQ (to the special queue
"notification.sample"). Also, every 60 seconds we run only
CPUPollster and send results to MQ again. After this, a
notification agent should read these messages and apply the
second "part" of pipeline.yaml, i.e. sinks. But how does this
agent determine what sink to apply to a Sample? The algorithm
is simple: we read the Sample from the "notification.sample"
queue and check the "counter_name" of it (note that we don't
care about "interval" on this stage). After, we send this
Sample to every sink if the corresponding source has the
satisfied rule in "meters" section. In the example above,
every 10 minutes we have Samples for all the possible meters,
including "cpu", i.e. every 10 minutes we apply cpu_sink and
cpu_delta_sink for the Sample from meter_source source (it's
actually wrong, because these sinks refers to cpu_source
source and in Juno we applied only meter_sink). Also, every 1
minute (your change) we have only one Sample with
"counter_name=cpu". And now we also apply all sinks again,
because "cpu" satisfies the "*" and "cpu" rules in the sources
description. That's why you see "cpu"-related samples every
minute. I assume that every 10 minute you may see two "cpu",
"cpu_delta" and "cpu_util"-related samples. </div>
<div> The main problem we have now in Ceilometer (starting
from Liberty) is that we have pipeline.yaml, but this file is
not fully applied neither on Polling agents (Central and
Compute), nor on Notification. On Polling, we are interested
only in sources and don't use transformers information; on
Notification, we don't care about intervals. </div>
<div> I don't see any workaround in your case, because you
need "cpu" samples for "cpu_util" construction, and you need
it every 60 seconds, i.e. you need to receive "cpu" every 1
minute and it's impossible to configure "please write down cpu
Sample to db only once in 10 minutes, ignore all other
occurrence of cpu Sample, use it only for transformation".</div>
<div><br>
</div>
<div> Sorry for bad news. I hope that this behaviour will be
discussed soon on the Summit. </div>
<div><br>
</div>
<div>Thanks,</div>
<div>Nadya</div>
<div><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Apr 8, 2016 at 5:03 PM,
rezroo <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:openstack@roodsari.us" target="_blank">openstack@roodsari.us</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Hi - I have been
using ceilometer in devstack for a while, and what I
used to be able to do is modify pipeline.yaml to set the
polling interval for "name: cpu_source" to 60, and then
publish it using udp. The diff for the changes are below
for the Liberty pipeline.yaml:<br>
<blockquote><small><i>stack@vlab:/etc/ceilometer$ diff
pipeline.yaml.bak pipeline.yaml</i><i><br>
</i><i>10c10</i><i><br>
</i><i>< interval: 600</i><i><br>
</i><i>---</i><i><br>
</i><i>> interval: 60</i><i><br>
</i><i>53c53</i><i><br>
</i><i>< - notifier://</i><i><br>
</i><i>---</i><i><br>
</i><i>> - udp://<a
moz-do-not-send="true"
href="http://127.0.0.1:4952" target="_blank">127.0.0.1:4952</a></i><i><br>
</i></small></blockquote>
The desired and net effect of these changes in Juno was
the I would get cpu_util samples every minute, but cpu
meters every 10 minute, in the Juno release.<br>
<br>
These same changes in Liberty are not having the same
effect. I get cpu, cpu_util, and cpu.delta every 1
minute. Why are cpu samples arriving every minute?<br>
<br>
<blockquote><i><small>stack@lab:~$ ceilometer
sample-list -m cpu | head</small></i><i><br>
</i><i><small>+--------------------------------------+------+------------+------------+------+----------------------------+</small></i><i><br>
</i><i><small>| Resource ID |
Name | Type | Volume | Unit |
Timestamp |</small></i><i><br>
</i><i><small>+--------------------------------------+------+------------+------------+------+----------------------------+</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu | cumulative | 1.0746e+11 | ns |
2016-04-08T13:48:17.380281 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu | cumulative | 1.0739e+11 | ns |
2016-04-08T13:47:17.357667 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu | cumulative | 1.073e+11 | ns |
2016-04-08T13:46:17.414188 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu | cumulative | 1.0723e+11 | ns |
2016-04-08T13:45:17.356869 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu | cumulative | 1.0715e+11 | ns |
2016-04-08T13:44:17.357771 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu | cumulative | 1.0707e+11 | ns |
2016-04-08T13:43:17.349820 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu | cumulative | 1.0699e+11 | ns |
2016-04-08T13:42:17.348110 |</small></i><i><br>
</i><i><br>
</i><i><small>stack@lab:~$ ceilometer sample-list -m
cpu.delta | head</small></i><i><br>
</i><i><small>+--------------------------------------+-----------+-------+-------------+------+----------------------------+</small></i><i><br>
</i><i><small>| Resource ID |
Name | Type | Volume | Unit |
Timestamp |</small></i><i><br>
</i><i><small>+--------------------------------------+-----------+-------+-------------+------+----------------------------+</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu.delta | delta | 70000000.0 | ns |
2016-04-08T13:48:17.380281 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu.delta | delta | 90000000.0 | ns |
2016-04-08T13:47:17.357667 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu.delta | delta | 70000000.0 | ns |
2016-04-08T13:46:17.414188 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu.delta | delta | 80000000.0 | ns |
2016-04-08T13:45:17.356869 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu.delta | delta | 80000000.0 | ns |
2016-04-08T13:44:17.357771 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu.delta | delta | 80000000.0 | ns |
2016-04-08T13:43:17.349820 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu.delta | delta | 80000000.0 | ns |
2016-04-08T13:42:17.348110 |</small></i><i><br>
</i><i><br>
</i><i><small>stack@lab:~$ ceilometer sample-list -m
cpu_util | head</small></i><i><br>
</i><i><small>+--------------------------------------+----------+-------+----------------+------+----------------------------+</small></i><i><br>
</i><i><small>| Resource ID |
Name | Type | Volume | Unit |
Timestamp |</small></i><i><br>
</i><i><small>+--------------------------------------+----------+-------+----------------+------+----------------------------+</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu_util | gauge | 0.116622711567 | % |
2016-04-08T13:48:17.380281 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu_util | gauge | 0.150141435735 | % |
2016-04-08T13:47:17.357667 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu_util | gauge | 0.116555319427 | % |
2016-04-08T13:46:17.414188 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu_util | gauge | 0.133335337808 | % |
2016-04-08T13:45:17.356869 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu_util | gauge | 0.133315666786 | % |
2016-04-08T13:44:17.357771 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu_util | gauge | 0.133329533442 | % |
2016-04-08T13:43:17.349820 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b |
cpu_util | gauge | 0.133329311232 | % |
2016-04-08T13:42:17.348110 |</small></i><br>
</blockquote>
Is this a ceilometer bug, or change? Is my understanding
of the desired effect incorrect?<br>
Thanks,<br>
Reza<br>
</div>
<br>
_______________________________________________<br>
Mailing list: <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack"
rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a moz-do-not-send="true"
href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
Unsubscribe : <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack"
rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</body>
</html>