[openstack-dev] [nova][ceilometer] cputime value resets on restart/shutdown

gord chung gord at live.ca
Tue Sep 1 18:20:25 UTC 2015



On 01/09/2015 12:10 PM, Julien Danjou wrote:
> On Tue, Sep 01 2015, gord chung wrote:
>
>> but if it's not actually cumulative in Ceilometer (pre-storage), should we
>> really be tagging it as such?
> We only have 3 meters type, and the cumulative definition I wrote
> somewhere back in 2012 states that it can reset to 0. Sorry. :-)

we shouldn't redefine real words -- no one reads definitions for words 
they already know... also, i'm not sure the 'reset to 0' definition is 
easily accessible (i'm not sure where it exists) :-\

>
>> so i was thinking rather than modify the existing meter, we build a new
>> cpu.delta meter, which is gives the delta. it seems like giving a delta meter
>> would make the behaviour more consistent.
> …with data loss if you restart the polling agent and it then ignores the
> previous value of the meter.
>
> Except if you connect the polling system to the previous state of the
> meter, which requires to either:
> 1. Connect the polling system to the storage system
> 2. Store locally the latest value you fetched
>
> Option 1 sounds crazy, option 2 sounds less crazy, but still hackish.
>
> Whereas having a system that can compute the delta afterward with all
> the value at its reach sounds way better – that's why I'm in favor of
> doing that in the storage system (e.g. Gnocchi).
>
i realise now we're just rehashing the same topic from 3 years ago[1]. 
it seemed like the issue then was we couldn't scale out multiple 
notification consumers... which we can now.

i agree that (1) is bad. (2) has potential data loss on agent restart... 
that said, you already have data loss regardless as when you poll 
'cumulative' meter, the cputime between poll cycles is loss during 
restart. if you handle this at storage level, every subsequent sample 
will be be off by an arbitrary amount after each restart where the delta 
solution, you'll only be off a single sample whenever instance/agent 
restarts.

[1]http://lists.openstack.org/pipermail/openstack-dev/2012-November/003297.html

-- 
gord




More information about the OpenStack-dev mailing list