For embedded mode, defer initialisation to first request requiring it.

[GoogleCodeExporter]

The mod_wsgi module originally initialised Python in Apache parent process. 
This is because that is what mod_python did. In version 3.X that changed, to 
being deferred to child processes due to memory leaks in Python when Python 
shutdown, library unloaded and reloaded. Using WSGIRestrictEmbedded you could 
prevent Python being initialised at all in child processes so long as authnz 
hooks not used. Despite this, even when people use daemon mode only they don't 
use WSGIRestrictEmbedded and so benefit from it not being seen.

The alternative is to defer Python initialisation in embedded mode processes 
until first request arrives that requires it. That way doesn't matter whether 
WSGIRestrictEmbedded not set, if daemon mode only used, then Python wouldn't 
get initialised through virtue of no request being handled in embedded process 
that requires it.

Deferring Python initialisation to first request would also be useful for using 
embedded mode with peruser MPM which forks process for each distinct request so 
can run as specific user id with process being reclaimed afterwards. Those 
requests may not even require Python though, so right now initialising Python 
for no reason. By deferring to when actually needed, these wouldn't be 
affected. You still end up seeing Python initialised on each request that needs 
it, but that is better than it being done on every request when peruser forks 
process and Python isn't even needed.

Original issue reported on code.google.com by [email protected] on 27 Jan 2011 at 9:55

[GoogleCodeExporter]

Note that if preloading setup with WSGIImportScript or 
process-group/application-group options to WSGIDaemonProcess, then should still 
be on process startup.

Original comment by [email protected] on 27 Jan 2011 at 9:56

[GoogleCodeExporter]

Possible problem with all this though is that code depends on fact that Python 
interpreter initialisation is done from main Apache thread and that shutdown of 
Python is then later done from same thread. In deferred initialisation, Python 
will be initialised in a request thread, but then shutdown would be done in 
main interpreter thread which had never called into the Python interpreter 
before and so will not have an active thread state object. May just need to 
initialise one first, but overall not sure if Python is going to do strange 
things if shutdown from different thread than the one it was initialised from.

Original comment by [email protected] on 27 Jan 2011 at 10:00

[GoogleCodeExporter]

Adding a patch which is a proof of concept to achieve true lazy loading which 
achieves the above except correct thread synchronization for the 
initialization. Adding this should be as simple as moving it below the mutex 
locker and changing mutex initialization so that it occurs *before* the 
initialization is done (since it currently initializes that mutex inside it.)

Also heads up for other problems like the shutting down from different thread 
potential issue mentioned by graham above.

Original comment by [email protected] on 27 Jan 2011 at 11:24

Attachments:

• mod_wsgi_true_lazy_load.patch