Why whitelists (and blacklists) are long-term poison for online systems

December 7, 2009

Every so often, well intentioned people propose that we can deal with the problems of various sorts of abuse in online forums and systems by having people build whitelists or blacklists (often with some sort of automated web of trust system). While this sounds great, it's a bad idea.

One of the old lessons from Usenet is that resorting to individual whitelists or blacklists to make a forum usable is ultimately destructive to the forum itself, because it more or less totally kills off the supply of good newcomers to the forum. Newcomers to your forum have no such list, so they experience the place in its full raw and unfiltered form. Almost invariably, this has one of two effects; either the newcomer is driven away entirely by the noise and chaos, or they become corrupted by what they rationally perceive as the local culture and turn themselves into yet another one of the trolls and vandals that are adding to the noise.

(Note that it does no good to send your filter to new participants in the forum; many people will take one look at the sewer and go away without posting anything or registering or whatever.)

Usenet unfortunately not just taught this lesson but illustrated this process quite graphically. Many newsgroups followed a predictable descent into noise; various sorts of problem people started the downhill slide, current participants moved to killfiles to preserve what they could, good newcomers stopped showing up, and over time more and more of the good oldbies left without any new blood to take their place. The result has been wasteland after wasteland (and people sadly pining for the 'good old days', which may or may not have ever existed).

As a corollary, this generalizes to any scheme where you must have some magic knowledge in order to make it work, such as webs of trust in cryptography. Somehow you have to bootstrap new people with the special knowledge that they need. If you rely on lucky contact with people, many people will drop through the cracks. If you rely on a central starter seed, in practice you might as well have an entirely central system by default because that's what you're going to wind up with. (And that default central system will be just as vulnerable to capture as ever.)

So are whitelists and blacklists pointless? Not at all, of course. They are a decent way of saving what you can from an existing system that is already going to heck. But you can't pretend that they are going to save the system itself; by the time you need such lists, the overall system is probably doomed, barring some heroic measure to change the entire culture and drive the noise out.

(Depressing applications to email are left as an exercise for the reader.)

(Disclaimer: since I learned this from Usenet, it is in no way original to me.)

Written on 07 December 2009.
« Overcoming the drawbacks of preforking accept() servers
Why I am not enthused about etckeeper and similar systems »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Dec 7 00:52:45 2009
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.