Open Stack

On 05/21/2013 05:55 AM, Julien Danjou wrote:
> On Mon, May 20 2013, Sandy Walsh wrote:
> 
> Hi Sandy,
> 
>> 1. Data comes in from somewhere (an Event)
> 
> I wouldn't call polling an event, but I get the idea.
> 
>> It would probably even be possible to use a format that could auto-generate the docs for these Meters.
>>
>> This could dramatically reduce the amount of code in Ceilometer today and
>> greatly increase the ability for installers to customize it without code
>> changes.
>>
>> Thoughts?
> 
> I think that could work for a lot of meters indeed. I didn't check which
> ones would fail, but that might be acceptable.
> 
> That may sound even to be the good direction to me.

Cool ... if the rest of the feedback is positive I'll convert this into
a blueprint.

> 
> With your Event recording work on the way, I would envision a future
> where we could avoid entirely the Meter storage and compute the meters
> dyamically via the API based on Events. So we would store only Events.
> Meaning that if we "transition" our current polling code to passing via
> an intermediate Event format before Meter, we're preparing for the
> future.

I'd love to think this is feasible in a practical sense and if it could
all be done in the db it might be. But based on the complexity of the
queries we're doing in StackTach the only way to get the results we need
in a timely fashion is to precompute much of it.

So, for simple Meters it might be possible, but for the more complex
ones (like .start/.end diff or api->last_service time) there's no way.

I think we really need to focus on the publishing pipeline and
optimizing how aggregation/rollups are done. Also, support for multiple
collectors while dealing with "settle-time" / jitter. Assume a Report
will be the most complex data structure we create.

But let me stew on the idea, it's intriguing. :)

-S


> 
> Thoughts? :)
>