Wandering Thoughts archives

The Apache mod_qos module worked for us

We run a shared web server where our users can run CGIs and so on in response to incoming HTTP requests. This presents several obvious potential problems, and recently we ran into one of them. A user had a quite slow CGI (although not a CPU-consuming one) and, as sometimes happens when your users are computer scientists sharing their hot research results, it got linked to from a popular place and a result the requests for it just poured in. In not very much time at all, the slow CGIs were using up all of Apache's request slots and nothing else could get a request in edgewise.

On the one hand, I didn't want to just turn off the user's CGI entirely. It's actively great that lots of people are interested in people's research results and we'd be serving our users very badly if we shut that down whenever it happened. On the other hand, it's a shared web server with other important things hosted on it, so I needed to keep the web server functioning in general. What I needed was something to limit how many concurrent requests for this particular CGI. Fortunately there is a (third-party) Apache module that can do this, ``mod_qos.

Mod_qos has a whole lot of configuration settings, most of which I didn't try to play with. What I used (and what worked) is the simple QS_LocRequestLimitMatch directive:

QS_LocRequestLimitMatch "^/(~|%7E|%7e)USER/.*$" NNN

Rereading the documentation now suggests that I could have used the simpler QS_LocRequestLimit directive, but at the time it wasn't clear to me if this was the right choice. I used the three-part match for '~' because the mod_qos specifically says that the directive applies to the unparsed URL and I wasn't sure quite how unparsed it meant.

(At the time I was in a mood to be basically sure with one change, because it was happening on a Saturday.)

Given that this situation may come up in the future, it would be sort of nice if we could set up generic per-URL resource limits or something. The module has the QS_LocRequestLimitDefault directive, but I don't know if it sets a global limit or a per-URL one. I'd have to experiment with this.

(In general, mod_qos seems like the kind of thing I should experiment with. It's potentially useful but fairly complex, and the documentation is clearly written for people who are already somewhat familiar with various terms of art and so on.)

What I like web templating languages for

In reaction to my entry on the problems with creating new (web) template languages, some of the people on Hacker News felt that the time of web templating languages was over due to various improvements in full programming languages. As it happens, I disagree. While there are situations where I would rather create HTML in code instead of through a template system, there are also plenty of situations where I would much rather use a template system.

In my opinion, templates work well in two interrelated situations. The first is when there is (much) more HTML than there is templating syntax to generate your dynamic content. One case this happens is if you're wrapping a form or some displayed (variable) information in a bunch of explanatory text. If you did this in the application's code, what you'd wind up with is mostly some large strings with periodic interruptions for the actual code; in my opinion, this is neither attractive nor very easy to follow (especially if the actual text of the page is broken up into multiple sections).

(I'd expect this to be even worse in a language that forced you to assemble the HTML through structured elements; you'd have a lot of code that existed just to make a whole stream of elements with the static content and string them together, interspersed with a few bits that generated the dynamic stuff.)

The second situation is where presenting the whole text and code of a page in one place makes it clearer what is going on and how the page is formed. Here, a single template is serving as the equivalent of a single giant function, except of course you probably wouldn't write a single giant function. Good template languages help this by creating compact but clear ways of describing their dynamic portions; generally these are much smaller than you could write in actual code without a lot of complex helper functions. While single templates can get tangled and complicated, this is in a sense a good thing because it's an honest expression of how complicated you're making generating the page be. Just as big, tangled functions are a code smell that suggests something needs to be refactored, big tangled template pages are probably a suggestion that they should be split into several template variants or otherwise restructured to be clearer.

Ultimately, templating languages versus programming languages are another incarnation of the gulf between shells and scripting languages. Programming languages are optimized for writing code that happens to have some embedded text, while templating languages are optimized for writing text that happens to have some embedded code. The more you have of one relative to the other, the more you will generally tilt one way.

(It's possible to make a templating system without embedded code, where your templates simply define places that code will later manipulate to add your custom content. I used to quite like this idea in the abstract, but I've come around to feeling that it's not what I want in practice.)