Open Stack

Yagi [1] is a really easy way to consume notifications and do stuff with them. We use it as the basis for our STv3 consumption. 

Very easy to write a Handler (fill in the handle_events() method) and you're off to the races. 

Yagi has a proper worker than can be daemonized and flexible logging. Spawn many for larger loads. 

Even easier, but not as battle tested, is notabene [2] ... here's an example of what it takes to consume notifications [3]

[1] https://github.com/rackerlabs/yagi
[2] https://github.com/StackTach/notabene
[3] https://github.com/stackforge/stacktach-notigen/blob/master/bin/event_consumer.py



________________________________________
From: Clint Byrum [clint at fewbar.com]
Sent: Thursday, February 12, 2015 2:28 PM
To: openstack-operators
Subject: Re: [Openstack-operators] [Ceilometer] Real world experience with      Ceilometer deployments - Feedback requested

Excerpts from George Shuklin's message of 2015-02-11 17:59:02 -0800:
> Ceilometer is in sad state.
>
> 1. Collector leaks memory. We ran it on same host with mongo, and it
> grab 29Gb out of 32, leaving mongo with less than gig memory available.

I wonder how hard it would be to push Ceilometer down the road of being
an OpenStack shim for collectd instead of a full implementation. This
would make the problem above go away, as collectd is written in C and is
well known to be highly optimized for exactly this type of workload.

You would need a more advanced AMQP plugin that understands how to turn
the notifications in OpenStack into collectd values, and then make some
decisions on whether to keep Ceilometer's SQL/MongoDB backend or just
teach Ceilometer to read from the various collectd output formats. I
think the latter will be a bigger win, but the former would be easier
for a more incremental migration.

Anyway, if people are interested in "saving" Ceilometer from being a
bit sluggish, that seems like a good first step in the investigation.

_______________________________________________
OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators