The Apache mod_qos module worked for us
We run a shared web server where our users can run CGIs and so on in response to incoming HTTP requests. This presents several obvious potential problems, and recently we ran into one of them. A user had a quite slow CGI (although not a CPU-consuming one) and, as sometimes happens when your users are computer scientists sharing their hot research results, it got linked to from a popular place and a result the requests for it just poured in. In not very much time at all, the slow CGIs were using up all of Apache's request slots and nothing else could get a request in edgewise.
On the one hand, I didn't want to just turn off the user's CGI entirely. It's actively great that lots of people are interested in people's research results and we'd be serving our users very badly if we shut that down whenever it happened. On the other hand, it's a shared web server with other important things hosted on it, so I needed to keep the web server functioning in general. What I needed was something to limit how many concurrent requests for this particular CGI. Fortunately there is a (third-party) Apache module that can do this, mod_qos.
Mod_qos has a whole lot of configuration settings, most of which I didn't try to play with. What I used (and what worked) is the simple QS_LocRequestLimitMatch directive:
QS_LocRequestLimitMatch "^/(~|%7E|%7e)USER/.*$" NNN
Rereading the documentation now suggests that I could have used the simpler QS_LocRequestLimit directive, but at the time it wasn't clear to me if this was the right choice. I used the three-part match for '~' because the mod_qos specifically says that the directive applies to the unparsed URL and I wasn't sure quite how unparsed it meant.
(At the time I was in a mood to be basically sure with one change, because it was happening on a Saturday.)
Given that this situation may come up in the future, it would be sort of nice if we could set up generic per-URL resource limits or something. The module has the QS_LocRequestLimitDefault directive, but I don't know if it sets a global limit or a per-URL one. I'd have to experiment with this.
(In general, mod_qos seems like the kind of thing I should experiment with. It's potentially useful but fairly complex, and the documentation is clearly written for people who are already somewhat familiar with various terms of art and so on.)