<HTML>
<HEAD>
<TITLE>Re: [Openstack] Memory leaks from greenthreads</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>I agree its not trivial, but it might be worth it?<BR>
<BR>
Or at least shift the model toward something semi-equivalent (multiprocessing??)<BR>
<BR>
If the python community says though that better inherent support for eventlet will arrive (ie something in the core python) someday then I say leave it. Otherwise I guess if the problems increase to a intolerable amount then something might have to happen (hopefully earlier rather than later).<BR>
<BR>
On 2/29/12 2:32 PM, "Johannes Erdfelt" <<a href="johannes@erdfelt.com">johannes@erdfelt.com</a>> wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>On Wed, Feb 29, 2012, Joshua Harlow <<a href="harlowja@yahoo-inc.com">harlowja@yahoo-inc.com</a>> wrote:<BR>
> Just a thought I was having, that others might want to chime in on.<BR>
><BR>
> Has there been any thinking around only using eventlet/greenlet for<BR>
> webserver endpoints and using something like multiprocessing for<BR>
> everything else?<BR>
><BR>
> I know its a fundamental change, but it would force people to think<BR>
> about how to break up there code into something that would work with<BR>
> a message passing architecture (this is already happening with<BR>
> nova + rabbitmq). Nova is a good example, but my thought was to go even<BR>
> further and have anything that needs to run for a long time (ie a<BR>
> equivalent of a nova manager) that is shared inside a application also<BR>
> be a separate "process" with a queue for message passing. Then maybe<BR>
> eventlet/greenlet isn't needed at all? This would force good<BR>
> interfaces, and we wouldn't have to worry about missing a monkey patch.<BR>
> Maybe the python people plan for multiprocess to replace<BR>
> eventlet/greenlet in the end anyway???<BR>
<BR>
I personally would be for an effort to remove eventlet. It does a pretty<BR>
good job, but monkey patching is error prone. For instance, my patch for<BR>
this eventlet memory leak also had to workaround lockfile patching the<BR>
threading.Thread object.<BR>
<BR>
The biggest problem is how the exception context gets cleared if<BR>
eventlet threads get switched. It's very hard to determine what<BR>
exception handlers need a sprinkling of save_and_reraise_exception() to<BR>
workaround that issue. That isn't the only reason for that function,<BR>
but it's difficult to know when it's necessary.<BR>
<BR>
That said, the work won't be trivial, especially with things like<BR>
connection pooling for XenAPI connections.<BR>
<BR>
JE<BR>
<BR>
<BR>
_______________________________________________<BR>
Mailing list: <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><BR>
Post to : <a href="openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><BR>
Unsubscribe : <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><BR>
More help : <a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a><BR>
<BR>
</SPAN></FONT></BLOCKQUOTE>
</BODY>
</HTML>