[openstack-dev] [Ceilometer][Oslo] Consuming Notifications in Batches
julien at danjou.info
Fri Dec 20 21:43:15 UTC 2013
On Fri, Dec 20 2013, Herndon, John Luke wrote:
> I think there is probably a tolerance for duplicates but you’re right,
> missing a notification is unacceptable. Can anyone weigh in on how big of a
> deal duplicates are for meters? Duplicates aren’t really unique to the
> batching approach, though. If a consumer dies after it’s inserted a message
> into the data store but before the message is acked, the message will be
> requeued and handled by another consumer resulting in a duplicate.
Duplicates can be a problem for metering, as if you see twice the same
event it's possible you will think it happened twice.
As for event storage, it won't be a problem if you use a good storage
driver that can have unique constraint; you'll just drop it and log the
fact that this should not have happened, or something like that.
> There will be situations where the message can’t be parsed, and those
> messages can’t just be thrown away. My current thought is that ceilometer
> could provide some sort of mechanism for sending messages that are invalid
> to an external data store (like a file, or a different topic on the amqp
> server) where a living, breathing human can look at them and try to parse
> out any meaningful information. Other errors might be “database not
> available”, in which case re-queing the message is probably the right way to
> go. If the consumer process crashes, all of the unasked messages need to be
> requeued and handled by a different consumer. Any other error cases?
Sounds good to me.
# Free Software hacker # independent consultant
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 832 bytes
Desc: not available
More information about the OpenStack-dev