The problem with noise

September 16, 2012

The problem with noise is that humans habituate to it very fast. This is a problem because humans see (and hear, and generally perceive) what they expect to see (hear, etc). If you are habituated to noise you hear noise and you ignore it. We are extremely badly adopted to picking up the one time out of many when what we are hearing is actual signal instead of noise.

(There are a number of fascinating and sometimes disturbing psychology experiments that demonstrate just how much of what we think we're perceiving is faked by our mind and how many things we overlook. I'll let you do the Internet searches for things like 'selective attention' yourself.)

In short, people are terrible at detecting true positives in a sea of false positives.

Any system that mostly generates false positives or other forms of noise that can't immediately be told from actual important things is in trouble; the more noise there is, the more trouble. In a relatively strong way, such systems are useless. By extension, creating a system that generates a lot of noise mixed in with its moderate signal and then saying 'but the information is there if you pay attention' is not solving the real problem, as usual. Engineering in the real world requires understanding the limitations of the real humans who will be using your system.

This applies all over the computing world, for example to security alerts, and underlies many other problems.

(By the way, you're still in trouble even if your signal is relatively distinct from noise. People do what they've been habituated to do and if that is 'delete email' or 'ignore phone', well, there you go. It doesn't really matter that people could easily tell that this was different if they paid attention to the email message because they probably won't, not if you've deluged them with noise before.)

(This is not novel observation and I've touched on this issue before in other entries. I just feel like writing the core issue down explicitly for once.)

Written on 16 September 2012.
« Sensible reboot monitoring
The options problem in Python »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Sep 16 00:42:27 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.