Wandering Thoughts archives

The history of commercial Unix and my pragmatism

I've said in the past that I'm a Unix pragmatist instead of a purist (and in general I've wound up a pragmatist about computing in general). In thinking about it recently, it's struck me that it's probably very hard to be both an old Unix hand and a purist, so it's not entirely surprising that I've wound up this way. To put it simply, the history of Unix since the late 1980s is a history of beautiful and pure things getting utterly crushed and ground up by the world. Over and over again, ugly things won.

An incomplete list:

• beautiful CISC and RISC architectures were driven into the ground by the x86 juggernaut. Modern 64-bit x86 is kind of an okay architecture from what I understand, but 32-bit x86 has always been an ugly register-starved monster.

• beautiful server hardware and hardware designs were crushed under the PC-compatible juggernaut and its collection of ugly hacks (eg the A20 line, which is probably still being emulated by the chipset on your 2015 vintage server).

All of the Unix vendors were eventually driven into surrender on this. Oh, I think that Oracle will still sell you SPARC based hardware, but almost no one really wants it any more because it very much lost on cost and performance.

• cool, pioneering Unix companies like MIPSco and NeXT went under or were bought out. This culminated in Sun being bought by Oracle, marking the final coda on the Unix workstation dream (which had died years before if we're being honest).

• Unix itself went through a series of uglifications in the early 1990s. Sun took SunOS 3, made it SunOS 4, and then engineered a high-speed collision between SunOS 4 and System V R4 that gave us the unappealing Solaris 2. This stampeded many of the other Unix vendors into making OSF/1, which was even worse (it was based on Mach at a time when this was not a good choice, among other defects).
• Unix vendors did terrible things to software layers on top of Unix. See CORBA, DCE, Motif, and let's be honest, Sun wasn't doing much better here either for a long time.

  It would not be much of an exaggeration to say that almost every architectural decision that most Unix vendors made turned out to be a mistake.

• Plan 9 was a complete practical failure. The Unix vendors collectively turned their back on essentially everything that came from the very source of Unix itself, and so by and large did Unix users. Later versions of Research Unix (v8, v9, and v10) were similar failures earlier on, with almost nothing from them making it into mainstream Unix.

  (And some of the things that did make the transition were terribly mangled, cf Dennis Ritchie's famous remark that 'streams means something different when shouted'.)

• Unix vendors in general punted on improving Unix and actually evolving it to be more Unixy for most of the mid to late 1990s. Of course most of them were preoccupied with dying at the time, which didn't help much.

• Once the era of free Unixes got rolling the open source *BSDs clung desperately to past history, some of which was better off replaced, while Linux reinvented everything, not always very well. Everyone mostly ignored the lessons of Plan 9, late Research Unix, and often even past Unix experience, despite having a golden opportunity to incorporate it into their systems.

And this is a vastly incomplete list of such disappointments.

If you started out as a purist, the past 25+ years of Unix history have mostly been a series of disappointments (some of them fairly crushing). Pick a thing that you cared about and you've probably had your dreams and hopes dashed. I think it's really hard to stay a purist when you're let down so many times and you have to work with such a parade of increasingly disappointing stuff. You can burn out so much that you leave Unix entirely, or you can move slowly from being a passionate purist to being a resigned pragmatist.

(There are passionate purists who have stuck to their guns and still work in Unix, but I don't think that there are all that many of them. You need a lot of fortitude, determination, and sheer bloody-mindedness.)

As a result, I kind of envy and admire the recent Unix people who are passionate purists right now. From my position on the sidelines in an easy chair, I want to cheer them on and tell them 'cling to your idealism as long as possible before the world grinds you down with disappointments'. And who knows? Maybe the next decade or two of Unix will be great, full of technical wins for the passionate. I'd certainly be happy to see Unix beauty win for once.

Sidebar: When I realized that I was becoming a pragmatist

As it happens, I can fairly precisely pinpoint when I became aware that I was turning into a pragmatist. It was when I had the opportunity to run Plan 9 instead of Unix but decided to pass because it would be too much of a hassle, both because of the restrictive hardware needs and due to how fairly cut off from the Unix computing environment around me I'd be. I looked at just how many things I'd be going without and said 'nope, the pure appeal and beauty of Plan 9 is not enough to get me to do this'.

(There's a part of me that's always regretted that I took the easy way.)

Why people were enthused about gcc early on in its life

Rob Landley recently wrote a widely linked piece on the BSD/System V split (via). There are any number of parts of his piece that I disagree with, but today I want to talk specifically about the early history of gcc, or specifically why people were enthused about it in the late 1980s. The origin of gcc is pretty straightforward; Richard Stallman knew he needed a free C compiler to enable his goal of a completely free Unix, so he wrote one fairly early in the FSF's life. But that's not the reason most people became enthused about it after it got released; after all, at that point pretty much everyone already had a C compiler.

What they didn't have was a good compiler. As hard as it may be to believe now (in a world where everyone knows that a good part of your benchmark performance comes from the compiler), back in the late 1980s the CISC-based Unix vendors didn't really see their C compiler as a competitive advantage. By and large most people simply ported or used the basic Portable C Compiler that had originally come from Research Unix, maybe added some optimizations, and then called it a day. If it could compile Unix correctly and didn't run terribly badly, it was pretty much good enough for DEC (for Ultrix), Sun, SGI, HP, and so on. Devoting more than a bit of engineering resources to the C compiler was apparently too much of a hard sell. And of course if you were running real BSD Unix on your Vax you didn't even have the benefit of any compiler improvements DEC had made.

(This is less crazy than it may seem today. At the time probably the major locus of performance oriented computing was in Fortran code; the vendors did tend to have good Fortran compilers, which they generally wanted extra money for. And remember that C was a relatively new language in the 1980s.)

Then along came gcc. Gcc was rather more modern than common Unix C compilers and even in its early state it often generated clearly better (ie faster) code on Vaxes and M68K machines. In the academic environments I hung around in, it soon became common knowledge (or at least folklore) that recompiling programs like the X server with gcc would get you visible performance improvements. Since the machines back then were not exactly fast, 'CC=gcc' started appearing in more and more Makefiles, configuration systems started preferring it when it was found, and so on. That you could basically get a performance improvement for free just by installing gcc made all sorts of people quite interested and enthused about it.

(The other attractive thing about gcc for garden variety people was that it often supported a more modern dialect of C than the normal Unix compiler. On top of that, a newer and thus better version of gcc was often just a ftp fetch and rebuild away; you didn't have to wait for an entire OS update.)

This gcc advantage was only an advantage on the common CISC architectures. RISC based Unix systems had much better compilers from the get go and if I remember right gcc version 1 actually had architectural limitations that meant it couldn't really do the advanced optimizations that vendor compilers did. Thus the irony of people switching to gcc when Sun yanked the free compilers from later versions of their OS was that they were going backwards in performance. And yes, we knew this full well. But free beat non-free for many people, especially at universities with limited budgets.

(People who used MIPS based systems from MIPS and then SGI were in luck, because I believe that SGI basically always bundled the generally excellent MIPSco compilers (perhaps only for universities). We didn't use gcc much on either our MIPS Ultrix machines or on the SGI machines that succeeded them.)

Update: There's more good stuff in this Hacker News comment and comment thread.