One of the famous computer programming quotes is 'premature optimization is the root of all evil' (C.A.R. Hoare quoted by Donald Knuth; attribution dammit (tm)).

A related issue is 'think about the scale of what you're planning'. A recent LiveJournal story provides a lovely example of this. To quote from it:

An increasing number of companies (large and small) are really insistent that we ping them with all blog updates, for reasons I won't rant about.

LiveJournal gets 3 or more public posts a second. That's a third of a second per post that has to include all DNS lookups, connection setup, sending the HTTP or SOAP or XML or whatever the ping format is, and connection teardown. (Apparently none of the companies gave LiveJournal a streaming interface, where LJ could open a connection once and then feed in results.)

The LiveJournal people gave a couple of these companies what they asked for. None of them could keep up.

The companies probably don't have bad or buggy software. I'm sure it works fine for the current set of blogs that pings them, and even has room for future growth. They just didn't think about the scale of what they were asking for from LiveJournal, and it probably didn't even occur to them to think about it.

Of course that's part of the problem of scale: it rarely occurs to people to think about it. Especially people almost never think about radical scale changes, whether up or down. This can lead to perfectly good solutions that don't fit the scale of the problem, or (as in this case) perfectly good solutions that don't quite handle the scale of a new problem.

When I start thinking about a system, I've found it useful to think about the scale of things as well as the problem itself. Sometimes this means I have to do more work; not infrequently it means I can do less. Thereby avoiding premature optimization and evil, and bringing me back to the quote up at the top.

Sidenote: the full optimization quote

A Google search wound up here, which cites the full quote as:

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil."

An MSDN web page gives the source as:

Quoted in Donald E. Knuth, Literate Programming (Stanford, California: Center for the Study of Language and Information, 1992), 276.