Wandering Thoughts archives

The standard trajectory of a field

Over and over again, fields of work have followed a common trajectory (both inside and outside of computers). At first, everything is new and unknown, and the field is pioneered by people who have to be innovators and often deeply technical. There are no known tools and procedures, so they need to be developed from scratch; generally everything has to be hand-crafted and unique and you spend a lot of time building all of this yourself.

But that's just the beginning. Over time this pioneering shifts and the field becomes increasingly routinized. As the problems become well explored and their solutions are increasingly well known, the field mostly or entirely turns into 'apply known solution Y to problem X', often with standard, already existing tools. The people in the field no longer need to be deeply technical innovators, because they are not solving new problems and building things from scratch. These days, the next step after routinization is automation (provided that it is easy enough to automate, or at least lucrative enough).

This shift is inevitable in most fields. As long as the problems and issues in the field do in fact repeat, people will necessarily gain familiarity with them over time; as this happens, people stop having to invent solutions to the problems on the spot and can instead just reuse the same solution as before (or at least some close variant of it). And it is usually less technically demanding to apply existing solutions than to invent new ones.

There is an important corollary to this: if you want to work on interesting things and with problems that have not been routinized, you need to be at the frontier somewhere. This implies that you need to keep moving. If you stay still in that comfortable bit of your field that you like, it is all but inevitable that the pioneering frontier will move past you and around you will spring up the cultivated fields of known problems with known, routine solutions. Then one day you will look up and wonder why you are spending all day applying recipes instead of inventing things.

Applications to the field of your choice are left as an exercise for the reader (or in the case of system administration, at least to another entry).