Open Stack

> As a curiosity, are there any ballpark numbers around the volume of
> notifications ceilometers can handle?  Thinking about large scale swift
> deployments I expect to see thousands or even 10s of thousands of events per
> second.  And that's with today's technology.  Looking longer term I wouldn't
> be surprised to see 100s or even millions per second.  And that's just
> swift.  I'd think other services not yet invented will have their own
> firehoses of data to contribute.

As things currently stand, 10s of thousands of events per second would
not be consumable by ceilometer, nor I'd suspect by the messaging bus
(open to correction on the latter point).

WRT "other services not yet invented", I think the approach used in the
swift case is fairly unique in that individual API calls are being metered,
as opposed to the existence and state of resources.

In the ceilometer swift middleware case, I feel we need to re-visit
the 1:1 mapping between API calls and notifications, and instead
batch up the API activity for individual tenants (and/or users)
over a longer time horizon before emitting a summary notification.

The existing ceilometer aggregation transformer in the local
pipeline may provide the pattern here, but probably requires some
specialization to accommodate multiple users PUT'ing to or
GET'ing from the same (tenant-scoped) object.

Cheers,
Eoghan

> -mark
> 
> -----Original Message-----
> From: Julien Danjou [mailto:julien at danjou.info]
> Sent: Thursday, July 31, 2014 5:24 AM
> To: Chris Dent
> Cc: OpenStack-dev at lists.openstack.org
> Subject: Re: [openstack-dev] [ceilometer] [swift] Improving
> ceilometer.objectstore.swift_middleware
> 
> On Wed, Jul 30 2014, Chris Dent wrote:
> 
> > What are other options? Of those above which are best or most
> > realistic?
> 
> I'm just thinking out loud and did not push that through, but I wonder if we
> should not try to use the oslo.messaging notifier middleware for that. It
> would be more standard (as it's the one usable on all HTTP
> pipelines) and rely on notification and generates events, as anyway, HTTP
> requests are events.
> Then it'd be up to Ceilometer to handle those notifications like it does for
> the rest of OpenStack.
> 
> --
> Julien Danjou
> /* Free Software hacker
>    http://julien.danjou.info */
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>