Open Stack

On 01/30/2013 02:49 PM, Doug Hellmann wrote:
> 
> 
> On Wed, Jan 30, 2013 at 8:32 AM, Julien Danjou <julien at danjou.info
> <mailto:julien at danjou.info>> wrote:
> 
>     On Tue, Jan 29 2013, Angus Salkeld wrote:
> 
>     > Nice walk through, I need to try digest how that is all going to
>     fit together.
>     >
>     > It initially seems to me that there is a new entity
>     "Notification/TracePoint"?
>     > They don't really seem like meters, and it might make sense to
>     just model
>     > correctly rather than trying to squeeze them into meters.
> 
>     +1
> 
>     From what I understand from your video, StackTach logs raw notifications
>     and then feed data into a few tables, doing some computing.
> 
>     So what it does is down to:
>     1. Stock raw notifications
>     2. Compute values from these notifications and cache them (= store them
>        into a few tables)
>        Some of these values might be meters.
> 
>     I don't think Ceilometer collector is going to store raw notifications.
>     There's no point into the whole Ceilometer project to store this data.
> 
> 
> I disagree. 
> 
> StackTach is answering questions we aren't, yet, like "how long did an
> operation take?", "what actually happened during processing of the
> request?", etc. I think we would benefit from adding at least some of
> that to ceilometer, particularly for SLA auditing. I agree with Angus'
> suggestion of adding new models, though, rather than trying to squeeze
> it into the existing data model as Sandy seems to be suggesting in the
> video. We would also likely want a new API endpoint for querying the new
> types of data.

I'm kind of torn actually. The video was just going with the proposal
that was currently on the table (Dimensions/Richer Metadata table) but I
do have performance concerns with this approach.

I originally was going with the New Datatypes approach as highlighted in
the wiki page. But I can see the ramifications of that with db
migrations under SQL. Each plugin would also have to deal with db
setup/teardown/migration, which is a pain. That said, the schema would
be much better for querying.

Tough problem. My gut says if the Metadata approach can perform (which
only testing can tell), it's an easier-to-maintain approach.

> My first suggestion was going to be to add a notification handler plugin
> that listened for the interesting notification messages and then stored
> them in the database, but if we are going to bring this into the core we
> should just add that step to the existing processing loop in the
> collector and then extend the storage engine API with a new method to
> store the notification.

Today I'll be digging in the code to learn what this means :)

> We may even be able to reduce some of the duplication of metadata if we
> just include a reference to the notification in the meter table instead
> of the entire metadata blob (I need to think more about whether that
> would actually work).

Yes! This!

-S

> Doug
>  
> 
> 
>     Your best call here is probably to keep that part of StackTach if you
>     really need all the raw notifications.
> 
>     For things are likely to be meter (i.e. things with a duration and/or a
>     volume), you can write some Ceilometer notification plugin building a
>     bunch of meter from what you get. Multi-publisher can also help you
>     building more advanced counter through the use of transformer if you
>     wish.
> 
>     --
>     Julien Danjou
>     ;; Free Software hacker & freelance
>     ;; http://julien.danjou.info
> 
>     _______________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>