Open Stack

On Thu, Dec 19, 2013 at 6:31 PM, Herndon, John Luke <john.herndon at hp.com>wrote:

> Hi Folks,
>
> The Rackspace-HP team has been putting a lot of effort into performance
> testing event collection in the ceilometer storage drivers[0]. Based on
> some results of this testing, we would like to support batch consumption
> of notifications, as it will greatly improve insertion performance. Batch
> consumption in this case means waiting for a certain number of
> notifications to arrive before sending to the storage
> driver.
>
> I¹d like to get feedback from the community about this feature, and how we
> are planning to implement it. Here is what I’m currently thinking:
>
> 1) This seems to fit well into oslo.messaging - batching may be a feature
> that other projects will find useful. After reviewing the changes that
> sileht has been working on in oslo.messaging, I think the right way to
> start off is to create a new executor that builds up a batch of
> notifications, and sends the batch to the dispatcher. We’d also add a
> timeout, so if a certain amount of time passes and the batch isn’t filled
> up, the notifications will be dispatched anyway. I’ve started a
> blueprint for this change and am filling in the details as I go along [1].
>

IIRC, the executor is meant to differentiate between threading, eventlet,
other async implementations, or other methods for dealing with the I/O. It
might be better to implement the batching at the dispatcher level instead.
That way no matter what I/O processing is in place, the batching will occur.


> 2) In ceilometer, initialize the notification listener with the batch
> executor instead of the eventlet executor (this should probably be
> configurable)[2]. We can then send the entire batch of notifications to
> the storage driver to be processed as events, while maintaining the
> current method for converting notifications into samples.
>
> 3) Error handling becomes more difficult. The executor needs to know if
> any of the notifications should be requeued. I think the right way to
> solve this is to return a list of notifications to requeue from the
> handler. Any better ideas?
>

Which "handler" do you mean?

Doug



>
> Is this the right approach to take? I¹m not an oslo.messaging expert, so
> if there is a proper way to implement this change, I¹m all ears!
>
> Thanks, happy holidays!
> -john
>
> 0: https://etherpad.openstack.org/p/ceilometer-data-store-scale-testing
> 1:
> https://blueprints.launchpad.net/oslo.messaging/+spec/bulk-consume-messages
> 2: https://blueprints.launchpad.net/ceilometer/+spec/use-bulk-notification
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131220/7eccbe27/attachment.html>