Overcoming the drawbacks of preforking accept() servers

December 5, 2009

I was going to say that while preforking accept()-based servers don't in practice have a thundering herd problem, they do have two other issues, namely there's no way to tell when you need to grow the number of processes you're using and no way to tell if a process has frozen. (The more complicated preforking scheme has neither problem.)

However, some thought showed me that it's possible to get around these problems while retaining the advantages of the pure accept()-based scheme. The key change to the simple version is that each child tells the master process when it goes idle and when it handles a new connection (doing so through a pipe that the master process set up for this purpose). Since it sees state transitions, the master process can now easily keep track of when a child has been busy on a single request for too long and kill it.

(Things will be more reliable under load if the child sends a timestamp in its messages, since the master may not process child messages immediately.)

Deciding when to reduce the work pool is relatively simple; the master process can keep a count of the minimum number of idle workers over the last N seconds. When this number gets high enough, it can either not restart workers when they exit after handling N requests or outright ask them to die (via a signal, for example).

Deciding when to spin up more workers is more challenging. The only approach I can think of is for the master process to monitor the server socket; if the socket has stayed accept()-able for some length of time and no worker process has changed its state, you start another one.

(The simpler approach to this is just to say that processes are cheap and so you will always start your final pool size, instead of trying to grow and shrink the worker pool dynamically. This didn't make sense for Apache but probably does these days for a backend server that is not worrying about talking to slow clients.)

Written on 05 December 2009.
« What version of Python is included in various current OSes
Why whitelists (and blacklists) are long-term poison for online systems »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Dec 5 23:30:17 2009
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.