iostat manpage documents
Show the percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.
It turns out that the manpage is wrong, which I found out by reading the kernel source because I was curious about what exactly it measured.
The actual definition of
%iowait is the percentage of the time that
the system was idle and at least one process was waiting for disk IO to
finish. (This is true for both the 2.6 kernel and Red Hat's special
2.4 kernels with better disk IO statistics, including Red Hat
(The actual kernel measure is the amount of time that each CPU has spent
in each mode; it shows up in
iostat converts this to
The difference may seem picky, but it's important because not all IO
causes processes to wait. For example, Linux doesn't immediately flush
data written to files to disk; it does it later, in the background,
when it's convenient. Under the manpage's definition, this background
flushing of data would take a system from
would slowly paging out unused bits of programs.
%iowait is roughly the amount of time that your system
could have been doing useful work if the disks were faster. A climbing
%iowait is a danger sign that your system may be running into an IO
bottleneck. A low iowait is not necessarily an indication that you
don't have an IO problem; you also want to look at things like the
number of processes shown as blocked ('
b' state) in
(Finding potential disk IO bottlenecks and troubleshooting them is a really big field, so this is in no way comprehensive advice.)