<html><body>
<p><font size="3" face=" Arial">Hi Stackers,</font><br>
<br>
<font size="3" face=" Arial">I'm working on the patch </font><a href="https://review.openstack.org/#/c/53270/"><font size="3" face=" Arial">53270 </font></a><font size="3" face=" Arial">for Swift profiling middleware and was blocked by a very annoying problem of the python eventlet profile which inherits from python standard profiler. It sometimes raised AssertionError of 'bad call' or 'bad return' in </font><font size="3" color="#990000" face="Consolas"><b>trace_dispatch_call</b></font><font size="3" face=" Arial"> and </font><font size="3" color="#990000" face="Consolas"><b>trace_dispatch_return</b></font><font size="3" face=" Arial"> hooked function. The eventlet profiler extend the return calls of standard profiler at line </font><a href="https://github.com/eventlet/eventlet/blob/master/eventlet/green/profile.py#L116"><font size="3" face=" Arial">116</font></a><font size="3" face=" Arial">. This problem will be gone if I change it back to python standard profile. So I guess it is specific to eventlet profile. It may not correctly handle some special cases. </font><br>
<br>
<font size="3" face=" Arial">The 2 places in the code of standard profiler that complains:</font><br>
<a href="https://github.com/python-git/python/blob/master/Lib/profile.py#L299"><font size="3" color="#4A6B82" face=" Arial">1) https://github.com/python-git/python/blob/master/Lib/profile.py#L299</font></a><br>
<font size="3" color="#00008B" face=" Arial"> assert</font><font size="3" face=" Arial"> rframe.f_back </font><font size="3" color="#00008B" face=" Arial">is</font><font size="3" face=" Arial"> frame.f_back, (</font><font size="3" color="#800000" face=" Arial">"Bad call"</font><font size="3" face=" Arial">, rfn, rframe, rframe.f_back, frame, frame.f_back)</font><br>
<br>
<font size="3" face=" Arial">2)</font><a href="https://github.com/python-git/python/blob/master/Lib/profile.py#L330"><font size="3" color="#4A6B82" face=" Arial"> https://github.com/python-git/python/blob/master/Lib/profile.py#L330</font></a><br>
<font size="3" color="#00008B" face=" Arial"> assert</font><font size="3" face=" Arial"> frame </font><font size="3" color="#00008B" face=" Arial">is</font><font size="3" face=" Arial"> self.cur[-</font><font size="3" color="#800000" face=" Arial">2</font><font size="3" face=" Arial">].f_back, (</font><font size="3" color="#800000" face=" Arial">"Bad return"</font><font size="3" face=" Arial">, self.cur[-</font><font size="3" color="#800000" face=" Arial">3</font><font size="3" face=" Arial">])</font><br>
<br>
<font size="3" face="Arial">I don't understand why to use assert there. What does it mean if it happens unexpected?</font>
<ul type="disc" style="padding-left: 22pt">
<li><font size="3" face="Arial">The profiler crashed and profiling results can't be used because of you have an unexpected call? (based on the practice of using assert statement to check your call contract)</font>
<li><font size="3" face="Arial">The results still can be used but not precise any more. you may need to catch the AssertionError.</font>
<li><font size="3" face="Arial">It is actually a bug need to be fixed here?</font></ul>
<br>
<font size="3" face="sans-serif">When I look into the eventlet profile, </font><font size="3" face="Arial">I'm also very curious about two line of code in eventlet profile module. It doesn't make sense to me since they are not reachable in any case. anybody can explain why?</font>
<ul type="disc" style="padding-left: 18pt">
<li><a href="https://github.com/eventlet/eventlet/blob/master/eventlet/green/profile.py#L103"><font size="3" color="#4A6B82" face="Arial">https://github.com/eventlet/eventlet/blob/master/eventlet/green/profile.py#L103</font></a><a href="https://github.com/eventlet/eventlet/blob/master/eventlet/green/profile.py#L110"></a>
<li><a href="https://github.com/eventlet/eventlet/blob/master/eventlet/green/profile.py#L110"><font size="3" color="#4A6B82" face="Arial"><u>https://github.com/eventlet/eventlet/blob/master/eventlet/green/profile.py#L110</u></font></a></ul>
<br>
<br>
<font size="3" face="sans-serif">Here's an example of stack trace:</font><br>
<font size="3" face="STHeitiSC-Light">-----------------------------------------------------------</font><br>
<font size="3" face="serif"><i>Traceback (most recent call last):</i></font><br>
<br>
<font size="3" face="serif"><i> File "/opt/stack/swift/bin/swift-proxy-server", line 23, in <module></i></font><br>
<font size="3" face="serif"><i> sys.exit(run_wsgi(conf_file, 'proxy-server', default_port=8080, **options))</i></font><br>
<br>
<font size="3" face="serif"><i> File "/opt/stack/swift/swift/common/wsgi.py", line 407, in run_wsgi</i></font><br>
<font size="3" face="serif"><i> run_server(conf, logger, sock)</i></font><br>
<br>
<font size="3" face="serif"><i> File "/opt/stack/swift/swift/common/wsgi.py", line 335, in run_server</i></font><br>
<font size="3" face="serif"><i> wsgi.server(sock, app, NullLogger(), custom_pool=pool)</i></font><br>
<br>
<font size="3" face="serif"><i> File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 693, in server</i></font><br>
<font size="3" face="serif"><i> client_socket = sock.accept()</i></font><br>
<br>
<font size="3" face="serif"><i> File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio.py", line 183, in accept</i></font><br>
<font size="3" face="serif"><i> timeout_exc=socket.timeout("timed out"))</i></font><br>
<br>
<font size="3" face="serif"><i> File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/__init__.py", line 155, in trampoline</i></font><br>
<font size="3" face="serif"><i> return hub.switch()</i></font><br>
<br>
<font size="3" face="serif"><i> File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch</i></font><br>
<font size="3" face="serif"><i> return self.greenlet.switch()</i></font><br>
<br>
<font size="3" face="serif"><i> File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 236, in run</i></font><br>
<font size="3" face="serif"><i> self.wait(sleep_time)</i></font><br>
<br>
<font size="3" face="serif"><i> File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/poll.py", line 113, in wait</i></font><br>
<font size="3" face="serif"><i> self.block_detect_post()</i></font><br>
<br>
<font size="3" face="serif"><i> File "/usr/lib/python2.7/profile.py", line 211, in trace_dispatch</i></font><br>
<br>
<font size="3" face="serif"><i> if self.dispatch[event](self, frame,t):</i></font><br>
<br>
<font size="3" face="serif"><i> File "/opt/stack/swift/swift/common/middleware/profile.py", line 239, in trace_dispatch_return_extend_back</i></font><br>
<font size="3" face="serif"><i> return self.trace_dispatch_return(frame, t);</i></font><br>
<br>
<font size="3" face="serif"><i> File "/usr/lib/python2.7/profile.py", line 312, in trace_dispatch_return</i></font><br>
<font size="3" face="serif"><i> assert frame is self.cur[-2].f_back, ("Bad return", frame.f_code, self.cur[-2].f_back.f_code)</i></font><br>
<br>
<font size="3" face="serif"><i>AssertionError: ('Bad return', <code object wait at 0x1a15030, file "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/poll.py", line 75>, <code object wait at 0x1191030, file "/usr/local/lib/python2.7/dist-packages/eventlet/queue.py", line 123>)</i></font><br>
<font size="3" face="STHeitiSC-Light">-----------------------------------------------------------</font><br>
<br>
<font size="2" face="sans-serif">Another paste example of stack trace is </font><a href="http://paste.openstack.org/show/DhIzYDvVVywpMX65Dp3a/"><font size="2" face="sans-serif">here</font></a><font size="2" face="sans-serif">.</font><br>
<br>
<font size="2" face="sans-serif">-Edward Zhang</font><br>
</body></html>