2006-08-16
The fun of 32-bit bugs
As computers (and disk space, and memory, and etc etc) get larger, the quantities that we deal with routinely get bigger too. And when they get bigger, fun things start happening.
Today's fun thing was that I doing some measurements of disk IO speed on a machine with 2 gigabytes of memory. My usual rule of thumb is to work on at least twice the amount of main memory to crush cache effects, which meant I was telling my benchmarking program to read and write 4 GB.
Which turned out to be kind of a problem, because I had declared a
variable as int instead of long (or better yet, off_t). At 4GB
it rolled over and various interesting things happened. I count myself
fortunately that it was instantly obvious that something was bad;
it could have just resulted in quietly wrong numbers that I might not
have noticed.
As our systems and what we do with them get bigger, I imagine I can look
forward to more and more incidents like this. Already, 2GB files are
becoming pretty common and it is more and more irritating when tools
don't deal with them, or don't deal well with them (some versions of
less will eat large files but mangle the percentages and jumping to
percentages, as I found out recently).
One benefit of 64-bit computing is that many of these problems can be
papered over on 64-bit platforms by a recompile, since that makes long
big enough again. (Some people will decry that as a quick fix.)
(Yes, technically this wasn't a 32-bit bug, it was a 31-bit bug. Close enough, says I.)
Sidebar: why not just use bonnie++ or the like?
Two reasons. First, bonnie++ benchmarks too much; at the moment I'm only interested in streaming read and write speeds. Second, bonnie++ was giving me odd results and I wanted to crosscheck them with something else.