[openstack-dev] [swift] [oslo] [osprofiler] [performance] OSprofiler in Swift

vinhnt at vn.fujitsu.com vinhnt at vn.fujitsu.com
Mon Jul 24 08:18:50 UTC 2017

Hello folks,

I'm sending this email for asking about the works in integrating OSprofiler into Swift.
Currently I'm working on this and the patches in Swift is waiting for review here: https://review.openstack.org/#/c/468316/

We knew that Swift has xprofile module [3] to do profiling works. However, OSprofiler is different from xprofile in many aspects [6].
xprofile is a profiling tool, OSprofiler is a distributed tracing tool for OpenStack services. We need OSProfiler in Swift for tracing across other OpenStack services.

FYI, OSprofiler provides functionality to generate a trace per request, that goes through all involve services.
This trace can visualize flow of a request [4] [5].
A trace from OSprofiler can help us know these things:
    - Performance bottle-neck of a service
    - Trouble-shooting issue in a service
    - Understanding flow of a request (from cli client or other client)
    - Trace can be store in persistent storage
    - Visualization trace flow in many OpenTracing compatible tracer [5] (will be done soon)

Hope that it will receive reviews from you all.

Some related references:
[1] OSprofiler documentation: https://docs.openstack.org/osprofiler/latest/
[2] Swift documentation: https://docs.openstack.org/swift/latest/ 
[3] Swift xprofile: https://docs.openstack.org/swift/latest/middleware.html#module-swift.common.middleware.xprofile 
[4] Demo with current OSprofiler patch set in Swift: https://tovin07.github.io/swift/swift-object-create.html
[5] A demo with OpenTracing compatible (using Uber Jaeger): https://tovin07.github.io/opentracing/jaeger-openstack-image-list.png
[6] Why not cProfile and others? https://docs.openstack.org/osprofiler/latest/user/background.html#why-not-cprofile-and-etc
[7] xprofile can use cProfile to profile internal python call: https://github.com/openstack/swift/search?utf8=%E2%9C%93&q=cprofile&type=
[8] Some concerns from notmyname: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift.2017-05-03.log.html#t2017-05-03T15:33:25
[9] Discussions from IRC meeting log: http://eavesdrop.openstack.org/meetings/swift/2017/swift.2017-06-14-07.00.log.html#l-152

Best regards,

Vinh Nguyen Trong
PODC - Fujitsu Vietnam Ltd.

More information about the OpenStack-dev mailing list