Rethinking avoiding Apache
Somewhat recently I wrote about when I'd use a web server other than Apache (despite Apache's temptations). I've recently discovered that I need to change those opinions somewhat; Apache turns out to be much more usable than I expected in a constrained resources situation.
One of my recent hobbies has been testing DWiki in a low-memory virtual machine (as I mentioned once in passing). I did my primary testing using nginx because it had an SCGI gateway, but with that working I decided on a whim to see how Apache plus mod_wsgi would do in the same small VM. To be honest, I expected Apache to explode spectacularly under any sort of real concurrent connection load, driving the virtual machine into the ground in the process.
To my total surprise, this did not happen. Not at all. Instead a more or less stock Ubuntu 12.04 Apache plus mod_wsgi setup handily dealt with all of the load I could throw at it. In my limited testing it was actually slightly faster on average than my nginx setup, dealt better with really extreme numbers of concurrent connections, and still left the machine with free memory. It was also easier to manage than my nginx lashup, which needed a separate system to run and restart the SCGI-based WSGI server that nginx talked to.
Part of this seems to be that Ubuntu 12.04 has sensible (ie small) Apache configuration settings. Another part is that mod_wsgi totally isolates the WSGI serving into separate processes (although they are still Apache processes). But regardless of all of this the whole setup just works and does so in an environment where I had previously expected Apache to be completely unsuitable. I am metaphorically eating my hat right about now.
(If I ever do deploy DWiki into such an environment, Apache plus mod_wsgi is now going to be my first choice. Not for performance, I doubt there's any meaningful practical difference, but because it's easier to manage because everything is in one spot and mod_wsgi has good support for easy code reloads.)
Sidebar: a caution about my performance results
Siege, the load tester I was using, reports only the average request time (and the maximum and minimum); it doesn't provide any difference about the distribution. It's possible that the distribution of response times is worse with Apache and the average is masking this. To do real testing I'd need to find a more thorough HTTP load tester (well, one with better stats reporting).