Why your fresh new memory pages are zero-filled

January 14, 2019

When you (or your programs) obtain memory directly from the operating system, you pretty much invariably get memory that is filled with zero bytes. The same thing is true if you ask for fresh empty disk space, on systems where you can do this (Unix, for example); by specification for Unix, if you extend a file without writing data, the 'empty space' is all 0 bytes. You might wonder why this is. The answer is pretty straightforward; the operating system has to put some specific value into the new memory and disk space, and people have historically picked all 0 bytes as that value.

(I am not dedicated enough to try to research very old operating system history to see if I can find the first OSes to do this. For reasons we're about to cover, it probably started no later than the 1960s.)

There is a story in this, although it is a short one. Once upon a time, when you asked the operating system for some memory or some disk space, the operating system didn't fill it with any defined value; instead it gave it to you with whatever random values it had had before. Since you were about to write to the memory (or disk space), the operating system setting it to a specific value before you overwrote it with your data was just a waste of CPU. This worked fine for a while, and then people on multi-user systems noticed that you could allocate a bunch of RAM or disk space, not write to it, and search through it to see if the previous users had left anything interesting there. Not infrequently they had. Very soon after people started doing this, operating systems stopped giving you new memory or disk space without clearing its old contents. The simplest way to clear the old contents is to overwrite them with some constant value, and apparently the simplest constant value (or at least the one everyone settled on) is 0.

(Since then, hardware and software have developed all sorts of high speed ways of setting memory to 0, partly because it's become such a common operation as a result of this operating system behavior. Some operating systems even zero memory in the background when idle so they can immediately hand out memory instead of having to pause to clear it.)

This behavior of clearing (new) memory to 0 bytes has had some inobvious consequences in places you might not think of immediately, but that's another entry.

Note that this is only what happens when you get memory directly from the operating system, generally with some form of system call. Most language environments don't return memory to the operating system when your code frees it (either explicitly or, in garbage collected languages, implicitly); instead they keep holding on to the now-free memory and recycle it when your code asks for more. This reallocated memory normally has the previous contents that your own code wrote into it. Although this can be a security issue too, it's not something the operating system deals with; it's your problem (or at least a problem for the language runtime).

Written on 14 January 2019.
« Two views of ZFS's GPL-incompatibility and the Linux kernel
Perhaps you no longer want to force a server-preferred TLS cipher order on clients »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jan 14 23:06:16 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.