Open Stack

On 01/30/2013 10:39 AM, Julien Danjou wrote:
> On Wed, Jan 30 2013, Sandy Walsh wrote:
> 
>> I don't understand the difference? Metering assumes archiving. If any
>> company with SEC requirements wants to use Ceilometer for billing, for
>> example, they'll have to store 90+ days of data.
>>
>> Can you elaborate on the ceilometer mandate?
> 
> Notifications aren't all about metering. Actually, they are called
> "notifications", not "meters". Things would be different if
> Nova/Glance/etc would emit directly meters, for example (hint hint).
> 
> Remember that Ceilometer is also providing an API endpoint to be
> queried, and what it stores via its `collector' is directly made to be
> used via its API.
> So the essence of the collector service is to treat and transform
> notifications into meters. To do metering.
> 
> Now, if what you want is to archive all notifications and build meters
> From that, there's a couple of solution, based on and around Ceilometer,
> but I don't think Ceilometer is going to be enough.
> 
> For example, you could have a smaller project just listening to
> notifications and storing them in a databse like StackTach does, and
> move the metering part into Ceilometer.
> Or you could write a driver notifier storing directly the notification
> into a database actually, I guess that depends on what you want to
> achieve.
> 
> But Ceilometer itself has no use of the raw notificatins once it has
> transformed them into meters.

Actually ... what if the json payload of the raw event was stored as
metadata on a Meter?

Is that permitted in Ceilometer?

If so, this gets us back to a bigger and more interesting question:

If we go with Metadata table + a Meter table we would have to index both
the Key and Value columns of the Metadata table.

For example:

Some metadata might be:

"event_name" : "compute.instance.run_instance" and
"request_id" : "xxxx-xxxx-xxxx-xxxx"

And we would want to see all <Foo> meters for request_id = 'xxxx-" and
event_name = "...."

Having something in Metadata other than primitive data types would
result in a storage nightmare.

Which also brings up the point about storing multiple types in the Value
column under SQL (no-sql is fine). It would have to be a tagged union /
variant record to say which value is being used:

Metadata
+ key : string
+ data_type : enum
+ integer_value : int
+ float_value : float
+ string_value : string
+ datetime_value : datetime
+ blob_value : memo
...

So, I guess we could apply indices only the meaningful data type columns.

Not sure how that would work with no-sql?

-S




-S