[openstack-dev] [Ceilometer][Oslo] Consuming Notifications in Batches
julien at danjou.info
Fri Dec 20 17:47:23 UTC 2013
On Fri, Dec 20 2013, Herndon, John Luke wrote:
> Yeah, I like this idea. As far as I can tell, AMQP doesn’t support grabbing
> more than a single message at a time, but we could definitely have the
> broker store up the batch before sending it along. Other protocols may
> support bulk consumption. My one concern with this approach is error
> handling. Currently the executors treat each notification individually. So
> let’s say the broker hands 100 messages at a time. When client is done
> processing the messages, the broker needs to know if message 25 had an error
> or not. We would somehow need to communicate back to the broker which
> messages failed. I think this may take some refactoring of
> executors/dispatchers. What do you think?
Yeah, it definitely needs to change the messaging API a bit to handle
such a case. But in the end that will be a good thing to support such a
case, it being natively supported by the broker or not.
For brokers where it's not possible, it may be simple enough to have a
"get_one_notification_nb()" method that would either return a
notification or None if there's none to read, and would that
consequently have to be _non-blocking_.
So if the transport is smart we write:
# Return up to max_number_of_notifications_to_read
Otherwise we do:
for i in range(conf.max_number_of_notifications_to_read):
notification = transport.get_one_notification_nb():
So it's just about having the right primitive in oslo.messaging, we can
then build on top of that wherever that is.
/* Free Software hacker * independent consultant
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 832 bytes
Desc: not available
More information about the OpenStack-dev