[openstack-dev] Problem with debugging program having eventlet.monkey_patch() code

Michael Basnight mbasnight at gmail.com
Mon Aug 27 14:05:50 UTC 2012


Thx dude i had no clue this existed! /magic

On Aug 27, 2012, at 7:17 AM, Jay Pipes wrote:

> You can also use the eventlet backdoor functionality already in Nova [1]
> 
> Just set the backdoor_port flag and telnet to it, then use it as a
> remote debugger using the following commands:
> 
> 	fo -- Find objects of a particular instance type
>        pgt -- Print the greenthreads stacks
> 
> For more on debugging eventlet with backdoor:
> 
> http://eventlet.net/doc/modules/backdoor.html
> 
> Best,
> -jay
> 
> [1]
> https://github.com/openstack/nova/blob/master/nova/common/eventlet_backdoor.py
> 
> On 08/24/2012 09:34 AM, Yong Sheng Gong wrote:
>> Another way is to modify monkey_patch() to monkey_patch(thread=False) in
>> single process mode. (nova-scheduler is a single processmode itself.)
>> 
>> Then u can debug it in pydev as usual.
>> 
>> 
>> -----Rafael Durán Castañeda <rafadurancastaneda at gmail.com> wrote: -----
>> To: openstack-dev at lists.openstack.org
>> From: Rafael Durán Castañeda <rafadurancastaneda at gmail.com>
>> Date: 08/22/2012 07:38PM
>> Subject: Re: [openstack-dev] Problem with debugging program having
>> eventlet.monkey_patch() code
>> 
>> On 08/21/2012 03:49 PM, michel.gauthier at bull.net wrote:
>>> I have installed winpdb, but I have still a problem while debugging
>>> nova-scheduler service :
>>> 
>>> on eventlet.monkey_patch()instruction, the debugger get the following
>>> exception :
>>> 
>>> 
>>> on winpdb GUI session ( launched thru : winpdb command and then
>>> atteched to the debugee session)
>>> Exception in thread Thread-57:
>>> Traceback (most recent call last):
>>>  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
>>>    self.run()
>>>  File "/usr/lib/python2.7/threading.py", line 504, in run
>>>    self.__target(*self.__args, **self.__kwargs)
>>>  File "/usr/local/lib/python2.7/dist-packages/rpdb2.py", line 10790,
>>> in __event_monitor_proc
>>>    (n, sel) =
>>> self.getSession().getProxy().wait_for_event(PING_TIMEOUT,
>>> self.m_remote_event_index)
>>>  File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
>>>    return self.__send(self.__name, args)
>>>  File "/usr/local/lib/python2.7/dist-packages/rpdb2.py", line 9520,
>>> in __request
>>>    raise _e
>>> RuntimeError: cannot release un-acquired lock
>>> 
>>> 
>>> on winpdb debugee session (launched thru : winpdb -d
>>> /usr/bin/nova-scheduler --flagfile=/etc/nova/nova.conf command)
>>> Exception in thread __worker_target:
>>> Traceback (most recent call last):
>>>  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
>>>    self.run()
>>>  File "/usr/local/lib/python2.7/dist-packages/rpdb2.py", line 4574,
>>> in run
>>>    threading.Thread.run(self)
>>>  File "/usr/lib/python2.7/threading.py", line 504, in run
>>>    self.__target(*self.__args, **self.__kwargs)
>>>  File "/usr/local/lib/python2.7/dist-packages/rpdb2.py", line 9361,
>>> in __worker_target
>>>    self.m_lock.release()
>>>  File "/usr/lib/python2.7/threading.py", line 142, in release
>>>    raise RuntimeError("cannot release un-acquired lock")
>>> RuntimeError: cannot release un-acquired lock
>>> 
>>> 
>>> Help for a solution.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> De :        Michael J Fork <mjfork at us.ibm.com>
>>> A :        OpenStack Development Mailing List
>>> <openstack-dev at lists.openstack.org>
>>> Date :        17/08/2012 15:03
>>> Objet :        Re: [openstack-dev] Problem with debugging program    
>>>   having        eventlet.monkey_patch() code
>>> ------------------------------------------------------------------------
>>> 
>>> 
>>> 
>>> We have found that Winpdb (_http://winpdb.org/_- platform independent
>>> despite Win in the name) works well with multi-threaded Python.  While
>>> using PyDev for development, run Winpdb as a remote debugger for the
>>> multithreaded service.  Flow is like this:
>>> 
>>> Launch Winpdb GUI
>>> # winpdb
>>> 
>>> Start Debugging
>>> # cd /home/openstack-dev/workspace/nova/bin
>>> # winpdb -d -r <file_to_debug>
>>> Set Password: openstack
>>> 
>>> Attach to Process
>>> 1) In Winpdb GUI, select File -> Attach
>>> 2) Enter password from step 4, should display files associated with
>>> that password
>>> 3) Select process/file then select OK
>>> 
>>> To Stop Debug or Make Changes and Restart
>>> 1) File -> Stop to detach current debugger
>>> 2) Make changes in Eclipse and Save (make sure development user has
>>> write privileges to repository)
>>> 3) kill and restart winpdb debugging process started above
>>> 4) Re-attach to the process in the Winpdb GUI
>>> 
>>> Michael
>>> 
>>> -------------------------------------------------
>>> Michael Fork
>>> Cloud Architect, Emerging Solutions
>>> IBM Systems & Technology Group
>>> 
>>> michel.gauthier at bull.net wrote on 08/17/2012 07:59:18 AM:
>>> 
>>>> From: michel.gauthier at bull.net
>>>> To: openstack-dev at lists.openstack.org,
>>>> Date: 08/17/2012 08:02 AM
>>>> Subject: [openstack-dev] Problem with debugging program having
>>>> eventlet.monkey_patch() code
>>>> 
>>>> Hi all,
>>>> I want to debug nova-scheduler service (Essex version)  code with
>>>> Eclipse->Pydev->Debug Configuration.
>>>> But when I launch it under Debug Configuration, I get the following
>>>> error messages :
>>>> 
>>>> pydev debugger: starting
>>>> Traceback (most recent call last):
>>>>  File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line
>>>> 336, in fire_timers
>>>>    timer()
>>>>  File "/usr/lib/python2.7/dist-packages/eventlet/hubs/timer.py",
>>>> line 56, in __call__
>>>>    cb(*args, **kw)
>>>>  File "/usr/lib/python2.7/dist-packages/eventlet/semaphore.py",
>>>> line 95, in _do_acquire
>>>>    waiter.switch()
>>>> error: cannot switch to a different thread
>>>> 
>>>> and then, Pydev debugger does not work!
>>>> 
>>>> In fact, these errors are due to the use of eventlet.monkey_patch()
>>> inside
>>>> nova-scheduler launcher (as well as inside other nova-* launchers).
>>>> 
>>>> Do you have any ideas to resolve this problem?
>>>> 
>>>> Thanks in advance._______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> 
>>> _http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev________________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> As a quick workaround you can change monkey patching at nova-scheduler
>> script:
>> 
>> eventlet.monkey_patch(thread=False)
>> 
>> HTH
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




More information about the OpenStack-dev mailing list