The downside of a unified buffer cache

July 20, 2007

When vendors started releasing Unix systems with a unified buffer cache, people discovered that the new operating system releases tended to perform not so well under load. Generally what happened is that programs got paged out madly, so any time you typed a new command line in your shell session there was a great big lurch as your shell was paged back in and then the program started and so on, and by the time you got back to your shell it had paged out again.

The underlying problem is that processes can use up pages through read() a lot faster than they can use up pages through virtual memory, because read() can touch a whole lot of pages in a single action while actual executing code only touches a page at a time. The net result was that programs doing disk IO basically ate the rest of the system; the effect that was particularly bad on multi-user systems, where something is doing disk IO pretty much all the time.

(Early versions of SunOS 4 exacerbated the situation by defaulting to maintaining only a remarkably small amount of free memory, so that merely starting a new command was virtually guaranteed to spark a page storm.)

Naturally, this was a hot topic for systems research in the early 1990s and produced a number of papers pointing out the issue, which was at least reassuring in that people understood why our (then) new systems were performing so badly, even if they didn't have a good solution.

(The situation has not gotten much better since then; virtual memory tuning problems still bedevil modern Unixes like FreeBSD and Linux. Many of the issues even look eerily familiar.)

Written on 20 July 2007.
« A safety tip: keep your different sorts of source trees separate
Weekly spam summary on July 21st, 2007 »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Jul 20 23:24:26 2007
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.