Open Stack

On 01/08/2016 04:44 AM, Radomir Dopieralski wrote:
> On 01/07/2016 05:55 PM, Mike Bayer wrote:
> 
>> but also even if you're under something like
>> mod_wsgi, you can spawn a child process or worker thread regardless.
>> You always have a Python interpreter running and all the things it can
>> do.
> 
> Actually you can't, reliably. Or, more precisely, you really shouldn't.
> Most web servers out there expect to do their own process/thread
> management and get really embarrassed if you do something like this,
> resulting in weird stuff happening.

I have to disagree with this as an across-the-board rule, partially
because my own work in building an enhanced database connection
management system is probably going to require that a background thread
be running in order to reap stale database connections.   Web servers
certainly do their own process/thread management, but a thoughtfully
organized background thread in conjunction with a supporting HTTP
service allows this to be feasible.   In the case of mod_wsgi,
particularly when using mod_wsgi in daemon mode, spawning of threads,
processes and in some scenarios even wholly separate applications are
supported use cases.

In mod_wsgi daemon mode (which is its recommended mode of use [1]), the
Python interpreter is not in-process with Apache in any case, and if you
set your own thread to be a "daemon", it won't block the process from
exiting.   I have successfully used this technique (again, carefully and
thoughtfully) to achieve asynchronous workers within Apache mod_wsgi
daemon-mode processes, without negative consequences.

Graham Dumpleton's own mod_wsgi documentation illustrates how to run a
background thread on development servers in mod_wsgi daemon mode in
order to achieve code reloading [2], and he also has produced a tool [3]
that uses a background thread in order to provide a debugging shell to a
running WSGI application which can work in either embedded or daemon
mode.

In [4], he illustrates using the WSGIImportScript mod_wsgi directive
under mod_wsgi daemon mode to actually spawn a whole docker container
when an Apache mod_wsgi process starts up; this isn't something I'd want
to do myself, but this is the author of mod_wsgi illustrating even
something as heavy as spinning up a whole docker instance under mod_wsgi
which then runs it's own WSGI process as a supported technique.

It is certainly reasonable that not all web application containers would
be effective with apps that include custom background threads or
processes (even though IMO any system that's running a Python
interpreter shouldn't have any issues with a limited number of
well-behaved daemon-mode threads), but at least in the case of mod_wsgi,
this is supported; that gives Openstack's HTTP-related applications with
carefully/thoughtfully organized background threads at least one
industry-standard alternative besides being forever welded to its
current homegrown WSGI server implementation.

[1] http://lanyrd.com/2013/pycon/scdyzk/

[2] https://code.google.com/p/modwsgi/wiki/ReloadingSourceCode

[3] https://github.com/GrahamDumpleton/ispyd

[4]
http://blog.dscpl.com.au/2015/07/using-apache-to-start-and-manage-docker.html