<div dir="ltr">Hello, Reza!<div><br></div><div>Thank you very much for this question. Your understanding is correct. In Liberty in Ceilometer we did a refactoring which has changed the behaviour in the way you've described. Let me try to explain in details. </div><div><br></div><div>Each cpu_util sample is the sample which is derived from cpu ones. In Juno, this transformation took place in Compute agent. In Liberty, we've started to do it in Notification agent. And it's the root cause of the issue. Let's consider the following pipeline.yam file:</div><div><table class="" style="border-collapse:collapse;border-spacing:0px;color:rgb(51,51,51);font-family:Helvetica,arial,nimbussansl,liberationsans,freesans,clean,sans-serif,'Apple Color Emoji','Segoe UI Emoji','Segoe UI Symbol';font-size:13px;line-height:18px"><tbody style=""><tr style=""><td id="LC2" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"><span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">sources:</span></span></td></tr><tr style=""><td id="L3" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC3" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="color:rgb(99,163,92)">name:</span> <span class="" style="">meter_source</span></span></td></tr><tr style=""><td id="L4" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC4" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(0,134,179)"><span class="" style="color:rgb(99,163,92)">interval:</span> 600</span></td></tr><tr style=""><td id="L5" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC5" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">meters:</span></span></td></tr><tr style=""><td id="L6" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC6" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style=""><span class="" style="">"</span>*<span class="" style="">"</span></span></span></td></tr><tr style=""><td id="L7" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC7" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">sinks:</span></span></td></tr><tr style=""><td id="L8" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC8" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="">meter_sink</span></span></td></tr><tr style=""><td id="L9" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC9" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="color:rgb(99,163,92)">name:</span> <span class="" style="">cpu_source</span></span></td></tr><tr style=""><td id="L10" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC10" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(0,134,179)"><span class="" style="color:rgb(99,163,92)">interval:</span> 60 <-----it's your change</span></td></tr><tr style=""><td id="L11" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC11" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">meters:</span></span></td></tr><tr style=""><td id="L12" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC12" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style=""><span class="" style="">"</span>cpu<span class="" style="">"</span></span></span></td></tr><tr style=""><td id="L13" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC13" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)"><span class="" style="color:rgb(99,163,92)">sinks:</span></span></td></tr><tr style=""><td id="L14" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC14" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="">cpu_sink</span></span></td></tr><tr style=""><td id="L15" class="" style="padding:0px 10px;width:50px;min-width:50px;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;color:rgba(0,0,0,0.298039);text-align:right;white-space:nowrap;vertical-align:top;border-style:solid;border-color:rgb(238,238,238);border-width:0px 1px 0px 0px"></td><td id="LC15" class="" style="padding:0px 10px;vertical-align:top;overflow:visible;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;word-wrap:normal;white-space:pre"> <span class="" style="color:rgb(24,54,145)">- <span class="" style="">cpu_delta_sink</span></span></td></tr></tbody></table></div><div> </div><div>It's written here that all possible pollsters (see setup.cfg <span style="color:rgb(0,0,0);font-family:Menlo;font-size:9pt">ceilometer.poll.compute section</span>) should be run every 10 minutes and "meter_sink" should be applied accordingly. Also, once in 60 seconds you want to run _only_ cpu pollster (<span style="color:rgb(0,0,0);font-family:Menlo;font-size:9pt">ceilometer.compute.pollsters.cpu:CPUPollster</span>) and transform the result as described in cpu_sink and cpu_delta_sink. In Juno, we did everything in Compute agent.</div><div> In Liberty the situation has been changed. In the example above, we run all pollsters once in 600 seconds and send the result to the RabbitMQ (to the special queue "notification.sample"). Also, every 60 seconds we run only CPUPollster and send results to MQ again. After this, a notification agent should read these messages and apply the second "part" of pipeline.yaml, i.e. sinks. But how does this agent determine what sink to apply to a Sample? The algorithm is simple: we read the Sample from the "notification.sample" queue and check the "counter_name" of it (note that we don't care about "interval" on this stage). After, we send this Sample to every sink if the corresponding source has the satisfied rule in "meters" section. In the example above, every 10 minutes we have Samples for all the possible meters, including "cpu", i.e. every 10 minutes we apply cpu_sink and cpu_delta_sink for the Sample from meter_source source (it's actually wrong, because these sinks refers to cpu_source source and in Juno we applied only meter_sink). Also, every 1 minute (your change) we have only one Sample with "counter_name=cpu". And now we also apply all sinks again, because "cpu" satisfies the "*" and "cpu" rules in the sources description. That's why you see "cpu"-related samples every minute. I assume that every 10 minute you may see two "cpu", "cpu_delta" and "cpu_util"-related samples. </div><div> The main problem we have now in Ceilometer (starting from Liberty) is that we have pipeline.yaml, but this file is not fully applied neither on Polling agents (Central and Compute), nor on Notification. On Polling, we are interested only in sources and don't use transformers information; on Notification, we don't care about intervals. </div><div> I don't see any workaround in your case, because you need "cpu" samples for "cpu_util" construction, and you need it every 60 seconds, i.e. you need to receive "cpu" every 1 minute and it's impossible to configure "please write down cpu Sample to db only once in 10 minutes, ignore all other occurrence of cpu Sample, use it only for transformation".</div><div><br></div><div> Sorry for bad news. I hope that this behaviour will be discussed soon on the Summit. </div><div><br></div><div>Thanks,</div><div>Nadya</div><div><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Apr 8, 2016 at 5:03 PM, rezroo <span dir="ltr"><<a href="mailto:openstack@roodsari.us" target="_blank">openstack@roodsari.us</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hi - I have been using ceilometer in devstack for a while, and what
I used to be able to do is modify pipeline.yaml to set the polling
interval for "name: cpu_source" to 60, and then publish it using
udp. The diff for the changes are below for the Liberty
pipeline.yaml:<br>
<blockquote><small><i>stack@vlab:/etc/ceilometer$ diff
pipeline.yaml.bak pipeline.yaml</i><i><br>
</i><i>10c10</i><i><br>
</i><i>< interval: 600</i><i><br>
</i><i>---</i><i><br>
</i><i>> interval: 60</i><i><br>
</i><i>53c53</i><i><br>
</i><i>< - notifier://</i><i><br>
</i><i>---</i><i><br>
</i><i>> - udp://<a href="http://127.0.0.1:4952" target="_blank">127.0.0.1:4952</a></i><i><br>
</i></small></blockquote>
The desired and net effect of these changes in Juno was the I would
get cpu_util samples every minute, but cpu meters every 10 minute,
in the Juno release.<br>
<br>
These same changes in Liberty are not having the same effect. I get
cpu, cpu_util, and cpu.delta every 1 minute. Why are cpu samples
arriving every minute?<br>
<br>
<blockquote><i><small>stack@lab:~$ ceilometer sample-list -m cpu |
head</small></i><i><br>
</i><i><small>+--------------------------------------+------+------------+------------+------+----------------------------+</small></i><i><br>
</i><i><small>| Resource ID | Name |
Type | Volume | Unit | Timestamp |</small></i><i><br>
</i><i><small>+--------------------------------------+------+------------+------------+------+----------------------------+</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu |
cumulative | 1.0746e+11 | ns | 2016-04-08T13:48:17.380281 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu |
cumulative | 1.0739e+11 | ns | 2016-04-08T13:47:17.357667 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu |
cumulative | 1.073e+11 | ns | 2016-04-08T13:46:17.414188 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu |
cumulative | 1.0723e+11 | ns | 2016-04-08T13:45:17.356869 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu |
cumulative | 1.0715e+11 | ns | 2016-04-08T13:44:17.357771 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu |
cumulative | 1.0707e+11 | ns | 2016-04-08T13:43:17.349820 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu |
cumulative | 1.0699e+11 | ns | 2016-04-08T13:42:17.348110 |</small></i><i><br>
</i><i><br>
</i><i><small>stack@lab:~$ ceilometer sample-list -m cpu.delta |
head</small></i><i><br>
</i><i><small>+--------------------------------------+-----------+-------+-------------+------+----------------------------+</small></i><i><br>
</i><i><small>| Resource ID | Name |
Type | Volume | Unit | Timestamp |</small></i><i><br>
</i><i><small>+--------------------------------------+-----------+-------+-------------+------+----------------------------+</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu.delta |
delta | 70000000.0 | ns | 2016-04-08T13:48:17.380281 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu.delta |
delta | 90000000.0 | ns | 2016-04-08T13:47:17.357667 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu.delta |
delta | 70000000.0 | ns | 2016-04-08T13:46:17.414188 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu.delta |
delta | 80000000.0 | ns | 2016-04-08T13:45:17.356869 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu.delta |
delta | 80000000.0 | ns | 2016-04-08T13:44:17.357771 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu.delta |
delta | 80000000.0 | ns | 2016-04-08T13:43:17.349820 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu.delta |
delta | 80000000.0 | ns | 2016-04-08T13:42:17.348110 |</small></i><i><br>
</i><i><br>
</i><i><small>stack@lab:~$ ceilometer sample-list -m cpu_util |
head</small></i><i><br>
</i><i><small>+--------------------------------------+----------+-------+----------------+------+----------------------------+</small></i><i><br>
</i><i><small>| Resource ID | Name |
Type | Volume | Unit | Timestamp |</small></i><i><br>
</i><i><small>+--------------------------------------+----------+-------+----------------+------+----------------------------+</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu_util |
gauge | 0.116622711567 | % | 2016-04-08T13:48:17.380281 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu_util |
gauge | 0.150141435735 | % | 2016-04-08T13:47:17.357667 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu_util |
gauge | 0.116555319427 | % | 2016-04-08T13:46:17.414188 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu_util |
gauge | 0.133335337808 | % | 2016-04-08T13:45:17.356869 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu_util |
gauge | 0.133315666786 | % | 2016-04-08T13:44:17.357771 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu_util |
gauge | 0.133329533442 | % | 2016-04-08T13:43:17.349820 |</small></i><i><br>
</i><i><small>| 6c4d5e4c-066e-4e98-b728-caaa66d8bf3b | cpu_util |
gauge | 0.133329311232 | % | 2016-04-08T13:42:17.348110 |</small></i><br>
</blockquote>
Is this a ceilometer bug, or change? Is my understanding of the
desired effect incorrect?<br>
Thanks,<br>
Reza<br>
</div>
<br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br></div></div>