Remember to think about the scale of things
One of the famous computer programming quotes is 'premature
optimization is the root of all evil' (C.A.R. Hoare quoted by Donald
Knuth; attribution dammit (tm)).
A related issue is 'think about the scale of what you're planning'.
A recent LiveJournal
provides a lovely example of this. To quote from it:
An increasing number of companies (large and small) are really
insistent that we ping them with all blog updates, for reasons I
won't rant about.
LiveJournal gets 3 or more public posts a second. That's a third of a
second per post that has to include all DNS lookups, connection setup,
sending the HTTP or SOAP or XML or whatever the ping format is, and
connection teardown. (Apparently none of the companies gave
LiveJournal a streaming interface, where LJ could open a connection
once and then feed in results.)
The LiveJournal people gave a couple of these companies what they
asked for. None of them could keep up.
The companies probably don't have bad or buggy software. I'm sure it
works fine for the current set of blogs that pings them, and even has
room for future growth. They just didn't think about the scale of
what they were asking for from LiveJournal, and it probably didn't
even occur to them to think about it.
Of course that's part of the problem of scale: it rarely occurs to
people to think about it. Especially people almost never think about
radical scale changes, whether up or down. This can lead to perfectly
good solutions that don't fit the scale of the problem, or (as in this
case) perfectly good solutions that don't quite handle the scale of a
When I start thinking about a system, I've found it useful to think
about the scale of things as well as the problem itself. Sometimes
this means I have to do more work; not infrequently it means I can do
less. Thereby avoiding premature optimization and evil, and bringing
me back to the quote up at the top.
Sidenote: the full optimization quote
A Google search wound up
cites the full quote as:
"We should forget about small efficiencies, say about 97% of the
time: premature optimization is the root of all evil."
An MSDN web page gives the source as:
Quoted in Donald E. Knuth, Literate Programming (Stanford,
California: Center for the Study of Language and Information, 1992),