Why System V shared memory may normally be limited

July 8, 2012

I recently read Robert Haas's Absurd Shared Memory Limits, in which he writes:

Despite the fact that System V shared memory is the only available shared memory implementation that provides this feature, most operating system vendors frown on it, and encourage users to use the newer POSIX shared memory facilities instead. They do this by limiting the amount of System V shared memory that can be allocated by default to absurdly small values, often 32MB.

To put it one way, I'm not sure that Unix vendors are being quite so calculating and deliberate here. System V shared memory has had low limits for basically as long as it's existed, so at a minimum OS vendors are just leaving things untouched through neglect (at the maximum, they're deliberately choosing not to increase the defaults to sane levels). There's a lot of this neglect in the kernel parameters for current Unixes, especially ones with a long history.

(Old time Unix vendors sometimes seem to act as if their customers would freak out if the vendors changed any tunable kernel parameters. This leads to absurd results given that memory, CPU speeds, and the number of CPUs in a system have all increased drastically over just the past decade. I've written about this before and it's somewhat of a hot button for me.)

However, my fallible memory suggests that there may have been another reason behind those low numbers, at least back in the day. Put simply, I believe that System V shared memory was initially not swappable. This would have made it doubly dangerous; it could be used by anyone, not just privileged processes, and these memory segments stayed around even after processes exited. In that situation, a quite low default limit makes a lot of sense.

(Of course, System V SHM is swappable on current systems and has been for years. If the low limits were for this reason, the reason's been inapplicable for, oh, a decade or two. But then we have that whole fossilization of tunable kernel parameters thing.)

Now, the disclaimer: I'm not sure that this was actually the case, and I can't find any sources online that would tell me either way. I may well be misremembering.

PS: even if current Unix vendors are leaving the System V SHM limits unchanged from the days of much smaller systems partly through a desire to have everyone move to Posix SHM, I suspect that this is more likely to be a policy of deliberate neglect than a policy of deliberately frustrating people.

Sidebar: how System V SHM could easily have been unswappable

In a modern Unix kernel design, leaving System V SHM unswappable would be a vaguely bizarre behavior. While SysV SHM is not directly tied to a process, there's any number of similar oddities that do get swapped and there's various bits of infrastructure that can be hijacked to help out.

However, System V SHM did not originate on modern Unix kernels; it originated on kernels that were so old and basic that they did not even have paging (just whole-process swapping). In this environment, swapping System V SHM segments would have required an entire new chunk of infrastructure, and all of the System V IPC mechanisms were basically quick hacks. I find it all too plausible that the initial implementation of System V SHM would simply have not bothered to build the extra kernel bits necessary to swap SHM segments.

(I do not have a high opinion of System V IPC in general, but that's for another entry.)

Written on 08 July 2012.
« Why Exim has a single queue for all email
Why the defaults for tunable kernel parameters matter »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Jul 8 01:06:15 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.