Feed Sign in with OpenID OpenID

Simon Willison’s Weblog

On mod_python

So, I’m getting stuck in to mod_python in a pretty big way at the moment. I’ve never even used mod_perl before, so coming from PHP this is turning out to be a real eye opener.

The key thing to realise about mod_python is that it gives you full control over the whole Apache request. Unlike PHP, you’re not writing scripts that just happen to use Apache as a go between—mod_python exposes a pretty big chunk of Apache’s internal API, and you are expected to hook straight in to it and start doing things. This certainly takes a while to get used to, but comes with some pretty big benefits. For one thing, your code is loaded once (well, once by each Apache process) and stays in memory. This means that global variables within your scripts persist between requests! Unfortunately, each Apache process (of which there are usually quite a few) has its own copy of the code so you can’t use this persistence ability for things like sessions, but it’s great for pooling database connections and keeping unchanging data such as templates in memory.

You can write your own handlers for the various parts of the Apache request. So far I’ve only been playing with the handler that serves up content, but it’s possible to do things like create a handler for the “Authentication” phase that authenticates users in some special way.

My biggest worry at the moment concerns the relative lack of documentation for mod_python on the web—the manual is pretty good, but other than that there’s precious little to get your teeth in to. Luckily the module has a reasonably high traffic mailing list which has so far answered all of the questions I’ve thrown at it promptly and in great detail. I hope to write more about mod_python as I become more comfortable with it. If anyone has any tips, I’d love to hear them.

This is On mod_python by Simon Willison, posted on 29th August 2003.

View blog reactions

Next: Fighting Filters and DDoS

Previous: Learning mod_rewrite

14 comments

  1. Count me in for being very interested in mod_python information.

    caiuschen - 29th August 2003 05:52 - #

  2. No fair, no fair! I've been wanting to find a mod_python host for a while so I can use it. I'm jealous.

    Keith - 29th August 2003 07:07 - #

  3. ...and fon't forget to check out jon's python modules that run on top of mod_python: These Python modules provide simple yet powerful multi-threaded object-oriented CGI/FastCGI/mod_python/html-templating facilities for the Python programming language.

    link: http://jonpy.sourceforge.net/

    dee - 29th August 2003 08:40 - #

  4. I would be a little careful about persisting database connections using mod_python and a forking Apache as you can potentially have an open database connection per Apache process. How serious that is depends on how you configure Apache.

    If you've got multiple applications running in Apache (virtual hosts maybe) then the above problem can be more severe - it's now possible to have one connection per application per Apache process.

    For those PHPers out there, it's a similar problem to using persistent connections.

    Matt Goodall - 29th August 2003 12:17 - #

  5. At some point could you relate why there's concern over mod_python?

    The most notable example I came across was:

    "The other ways I tried were mod_python and JonPy tools (which help with simple stuff like session management and db pooling) but just about *everyone* said "DON'T USE MOD_PYTHON!", I'm not even sure why, but I took them at their word." - Russell Beattie, http://www.russellbeattie.com/notebook/20030725.ht ml#125136

    DeanG - 29th August 2003 14:49 - #

  6. I've heard that mod_python has fallen out of favour recently, due to its embedding of n whole python interpreters in the apache runtime (where n is the number of processes, I believe). The better solution that was advanced was FastCGI (or was it SCGI?), where all the apache processes talked to a single python interpreter. There seems to be more information on a similar topic in this thread: http://mail.mems-exchange.org/pipermail/quixote-us ers/2003-February/001243.html

    Blake Winton - 29th August 2003 15:00 - #

  7. Mod_Python also will soon contain Python Server Pages :) I see globally scoped variables as a very bad thing, simply because you can't think in terms of a request anymore, which is bullocks. If you use a module in two places, the usages are not always sandboxed, etc. etc. etc.

    Sterling Hughes - 29th August 2003 15:41 - #

  8. Back in the day (when I still worked at fastengines.com, if anyone remembers them :) the "writing apache modules in C and perl" o'reilly book was actually a quite good explanation of the layers and api's -- and I'd look for something similar, even *without* a python-specific component, if I were to try doing this today; the key is getting perspective on what things are supposed to be like "inside", and which things to do at which "phase". That said, unless you actually *need* phase hooks (like clever authentication tricks) I think mod_perl (and by analogy, mod_python) are generally a mistake; fastcgi/speedycgi/whatever give you a lot of *reliability* wins, because you have a small clear interface, and easy ways to notice and clean up memory leaks, etc. by having your service be external. It also means you don't have to care about apache's multi-thread * multi-process model, you have whatever server set you want...

    Mark Eichin - 29th August 2003 19:24 - #

  9. I think Draco worths a look, it's very powerful: http://draco.boskant.nl/index.dsp

    Lawrence - 30th August 2003 00:12 - #

  10. Dean, from what I understand you have to be careful not to bring down your entire Apache server with mod_python, as it may have a reputation for instability. Also, I think I was being told this by a bunch of Twisted advocates as well... Hope that helps clear up why I wrote that. -Russ

    Russ - 30th August 2003 15:29 - #

  11. Wow Simon, you've got your first (that I've noticed) valid XHTML blog comment spam :)

    Swannie - 30th August 2003 17:00 - #

  12. Yeah...yeah, all this mod_python stuff is great, but are you aware that Scott Andrew went 2 days without saying "Fray Day"? Now that my friends is a story. Yad Yarf

    Yad Yarf - 31st August 2003 06:58 - #

  13. Russ, Thanks for the follow-up.

    Dean - 3rd September 2003 03:56 - #

Comments are closed.

Previously hosted at http://simon.incutio.com/archive/2003/08/29/modPython

A django site