<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" id="owaParaStyle">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">
<div>Oh, yes please! :)</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Kevin<br>
</div>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div id="divRpF234947" style="direction: ltr;"><font size="2" face="Tahoma" color="#000000"><b>From:</b> Ilya Shakhat [shakhat@gmail.com]<br>
<b>Sent:</b> Thursday, April 11, 2019 2:42 PM<br>
<b>To:</b> openstack-discuss@lists.openstack.org<br>
<b>Subject:</b> [osprofiler] Distributed tracing in OpenStack<br>
</font><br>
</div>
<div></div>
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Hi, <br>
</div>
<div><br>
</div>
<div>Distributed tracing is one of must-have features when one wants to track the full path of request going through different services and APIs. This makes it similar to shared request-id, but with nice visualization at the end [1]. In OpenStack the tracing
can be achieved via osprofiler library. The library was introduced 5 years ago, and back then there was no standard approach on how to do tracing and that's why it stays aside from what has become a mainstream. Yet there is no single standard, but the major
players are OpenTracing and OpenCensus communities. OpenTracing is represented by Uber's Jaeger which is the default tracer from k8s world. </div>
<div><br>
</div>
<div>Issues and limitations to be fixed:</div>
<div>1. Compatibility. While osprofiler library supports many different storage drivers, it has only one way of transferring trace context over the wire. Ideally the library should be compatible with other third-party tracers and allow traces to start in front
of OpenStack APIs (e.g. in user apps) and continue after (e.g. in storage systems, or network management tools). [2]</div>
<div>2. Operation mode. With osprofiler tracing is initiated by user request, while in industrial solutions the tracing can be managed centrally via dynamic sampling policies.<br>
</div>
<div>3. In-process trace propagation. Depending on execution model (threaded, async) the ways of storing current trace context differ. OSProfiler supports thread-local model, which recently got broken with new async implementation in openstacksdk [3]. With
OpenTracing it is possible to select the appropriate model alongside with tracer configuration.</div>
<div><br>
</div>
<div>What's the plan:</div>
<div>Switching to OpenTracing could be a good option to gain compatibility with 3rd-party solutions. The actual change should go to osprofiler library, but indirectly affects all OpenStack projects (should it be a global team goal then?). I'm going to make
a PoC of proposed change, so reviews would be highly appreciated.</div>
<div><br>
</div>
<div>Comments, suggestions?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Ilya<br>
</div>
<div><br>
</div>
<div>[1] e.g. <a href="http://logs.openstack.org/15/650915/4/check/tempest-smoke-py3-osprofiler-redis/7c6c14e/osprofiler-traces/trace-3e5cc660-8815-4079-86b9-778af8469d79.html.gz" target="_blank" rel="noopener noreferrer">
http://logs.openstack.org/15/650915/4/check/tempest-smoke-py3-osprofiler-redis/7c6c14e/osprofiler-traces/trace-3e5cc660-8815-4079-86b9-778af8469d79.html.gz</a><br>
</div>
<div>[2] <a href="https://bugs.launchpad.net/osprofiler/+bug/1798565" target="_blank" rel="noopener noreferrer">
https://bugs.launchpad.net/osprofiler/+bug/1798565</a></div>
<div>[3] <a href="https://bugs.launchpad.net/osprofiler/+bug/1818493" target="_blank" rel="noopener noreferrer">
https://bugs.launchpad.net/osprofiler/+bug/1818493</a><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>