Overcoming the drawbacks of preforking
I was going to say that while preforking
accept()-based servers don't
in practice have a thundering herd
problem, they do have two other issues, namely there's no way to tell
when you need to grow the number of processes you're using and no way to
tell if a process has frozen. (The more complicated preforking scheme has neither problem.)
However, some thought showed me that it's possible to get around these
problems while retaining the advantages of the pure
scheme. The key change to the simple version is that each child
tells the master process when it goes idle and when it handles a new
connection (doing so through a pipe that the master process set up for
this purpose). Since it sees state transitions, the master process can
now easily keep track of when a child has been busy on a single request
for too long and kill it.
(Things will be more reliable under load if the child sends a timestamp in its messages, since the master may not process child messages immediately.)
Deciding when to reduce the work pool is relatively simple; the master process can keep a count of the minimum number of idle workers over the last N seconds. When this number gets high enough, it can either not restart workers when they exit after handling N requests or outright ask them to die (via a signal, for example).
Deciding when to spin up more workers is more challenging. The only
approach I can think of is for the master process to monitor the server
socket; if the socket has stayed
accept()-able for some length of time
and no worker process has changed its state, you start another one.
(The simpler approach to this is just to say that processes are cheap and so you will always start your final pool size, instead of trying to grow and shrink the worker pool dynamically. This didn't make sense for Apache but probably does these days for a backend server that is not worrying about talking to slow clients.)