Open Stack

On Fri, Dec 20 2013, Herndon, John Luke wrote:

> I think there is probably a tolerance for duplicates but you’re right,
> missing a notification is unacceptable. Can anyone weigh in on how big of a
> deal duplicates are for meters? Duplicates aren’t really unique to the
> batching approach, though. If a consumer dies after it’s inserted a message
> into the data store but before the message is acked, the message will be
> requeued and handled by another consumer resulting in a duplicate.

Duplicates can be a problem for metering, as if you see twice the same
event it's possible you will think it happened twice.

As for event storage, it won't be a problem if you use a good storage
driver that can have unique constraint; you'll just drop it and log the
fact that this should not have happened, or something like that.

> There will be situations where the message can’t be parsed, and those
> messages can’t just be thrown away. My current thought is that ceilometer
> could provide some sort of mechanism for sending messages that are invalid
> to an external data store (like a file, or a different topic on the amqp
> server) where a living, breathing human can look at them and try to parse
> out any meaningful information. Other errors might be “database not
> available”, in which case re-queing the message is probably the right way to
> go. If the consumer process crashes, all of the unasked messages need to be
> requeued and handled by a different consumer. Any other error cases?

Sounds good to me.

-- 
Julien Danjou
# Free Software hacker # independent consultant
# http://julien.danjou.info
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131220/b2301fc7/attachment.pgp>