Open Stack

On 7/30/14, 8:06 AM, Chris Dent wrote:
>
> ceilometer/objectstore/swift_middleware.py[1] counts the size of web
> request and reponse bodies through the swift proxy server and publishes
> metrics of the size of the request and response and that a request
> happened at all.
>
> There are (at least) two bug reports associated with this bit of code:
>
> * avoid requirement on tarball for unit tests
>    https://bugs.launchpad.net/ceilometer/+bug/1285388
>
> * significant performance degradation when ceilometer middleware for
>    swift proxy uses
>    https://bugs.launchpad.net/ceilometer/+bug/1337761
>
> [snip]
 >
> Some options appear to be:
>
> * Move the middleware to swift or move the functionality to swift.
>
>    In the process make the functionality drop generic notifications for
>    storage.objects.incoming.bytes and storage.objects.outgoing.bytes
>    that anyone can consume, including ceilometer.
>
>    This could potentially address both bugs.
>
> * Move or copy swift.common.utils.{InputProxy,split_path} to somewhere
>    in oslo, but keep the middleware in ceilometer.
>
>    This would require somebody sharing the info on how to properly
>    participate in swift's logging setup without incorporating swift.
>
>    This would fix the first bug without saying anything about the
>    second.
>
> * Carry on importing the swift tarball or otherwise depending on
>    swift.
>
>    Fixes neither bug, maintains status quo.
>
> What are other options? Of those above which are best or most
> realistic?

Swift is already emitting those numbers[1] in statsd format; could 
ceilometer consume those metrics and convert them to whatever 
notification format it uses?

When configured to log to statsd, the Swift proxy will emit metrics of 
the form proxy-server.<type>.<verb>.<status>.xfer; for example, a 
successful object download would have a metric name of 
"proxy-server.object.GET.200.xfer" and a value of the number of bytes 
downloaded. Similarly, PUTs would look like 
"proxy-server.object.PUT.<2xx>.xfer".

If ceilometer were to consume these metrics in a process outside the 
Swift proxy server, this would solve both problems. The performance fix 
comes by being outside the Swift proxy, and consuming statsd metrics can 
be done without pulling in Swift code[2].

[1] 
http://docs.openstack.org/developer/swift/admin_guide.html#reporting-metrics-to-statsd

[2] e.g. pystatsd