[openstack-dev] Dealing with Popen and Eventlet

Joshua Harlow harlowja at yahoo-inc.com
Wed Nov 7 21:49:01 UTC 2012


Eck, I don't think a lot of deployment happens in a virtualenv (not 100% on this), a lot of people seem to like packages on real/vm hardware since openstack 'controls' a lot of the 'real hardware' in a way that venv doesn't 'hide' to well (if at all).

Yes it seems like u could use 'venv' to detect that eventlet is not installed via the installed site-packages, but I don't think this should be a deployment requirement when u need this kind of separation, a config setting seems applicable for those that want to use eventlet, or eventlet + venv, or no eventlet…

From: Dolph Mathews <dolph.mathews at gmail.com<mailto:dolph.mathews at gmail.com>>
Reply-To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Wednesday, November 7, 2012 12:43 PM
To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] Dealing with Popen and Eventlet

The naive approach (if eventlet is available, use it) just seems to reinforce that running in apache vs eventlet is a deployment issue (e.g. don't install eventlet, run in a virtualenv, etc).


-Dolph


On Wed, Nov 7, 2012 at 12:34 PM, Adam Young <ayoung at redhat.com<mailto:ayoung at redhat.com>> wrote:
On 11/07/2012 02:52 AM, Vishvananda Ishaya wrote:
On Nov 2, 2012, at 6:40 PM, Adam Young <ayoung at redhat.com<mailto:ayoung at redhat.com>> wrote:

Since Eventlet is based on continuations instead of threads or processes, any call that blocks is going to lock up the webserver. That is any I/O at all.  Thus, when trying to figure out how best to call the OpenSSL functions from Eventlet, I settled on what I thought would be the best tested approach: spin it off as a separate process.  And it seemed to work.

Well, it doesn't work.  The Eventlet library does not properly Monkey patch the subprocess calls from Python.  While this is something we should patch upstream in Eventlet,  we need something to deal with the existing Eventlet library for Grizzly development.

I've been trying to make it possible to run Keystone (and the other services eventually) in Apache.  Thus, the simple solution of replacing

import subprocess

with

import eventlet.green.subprocess

Won't work.  It solves the problem in Eventlet, but not apache.
Why not just do the standard

try:
   from eventlet.green import subprocess
except ImportError:
   import subprocess

We've used that in various places in nova before.

That logic is incorrect.  That  that logic is "if eventlet is available, use it." If you are trying to run Keystone in HTTPD, but eventlet happens to be installed because you are running say, nova on the same server, it will pick up eventlet from the site-libs. Thus, the logic we want is "If you are running in an eventlet server, use the eventlet subprocess."  A good rule of thumb is that the eventlet code should not be referenced from any files except those that explicitly choose to run eventlet.  In Keystone, that is the server.



Vish


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121107/3733c727/attachment.html>


More information about the OpenStack-dev mailing list