Wandering Thoughts archives

Some places where I think that Unix is incomplete or imperfect

A commentator on Unix needs to grow asked for some examples of what I had in mind and, well, the resulting comment got away from me.

This is a very incomplete list written off the top of my head (and is in no particular order). All of these issues are from the perspective of 'traditional Unix', the Unix on static servers that most everyone agrees is more or less fully Unixy. While modern Unixes have dealt with a number of them it's often not in ways that people really think are Unixy, so I consider them open problems that still need work.

• hotplug hardware (on basically everything), including issues like how /dev entries for new devices show up.

• on multi-user workstations (and then desktops), making it so that only the user on the console could use various bits of local hardware like the audio device. Pranking people by ssh'ing in to their workstation and then playing loud music is all well and good, but let's not.

  (A genuine multi-head environment makes this even less clear.)

• allowing the console user to access things like CD-ROMs, DVDs, USB memory sticks, and so on that they plug in to the hardware. Even apart from the inconvenience, 'su to root and run mount' is not a viable answer because in a multi-user workstation environment the console user may not have the root password. This needs to be controllable (some environments don't want to allow this) and is made more complicated because there are many different sorts of access that are needed beyond just mounting a filesystem (eg burning a DVD).

• dealing with the variable networking environment that a laptop or other mobile device sees as it moves around. Again, 'su to root' is a terrible answer here. This is more than just basic changes in network devices, IP addresses, and so on; it ripples through into user programs, DNS resolution, and any number of other areas.
• in general, handling various sorts of 'on-demand' and transient network connections.

• suspending and unsuspending machines, especially since there are often user-space things you want to do both on suspend and on unsuspend. This used to be just a laptop issue but is now also interesting for things like virtual machines.

• many aspects of networking in general. For example, what is a Unixy approach to firewalling, NAT, and so on, or does Unix have nothing to say about the right way to approach this? What is the right Unixy API for programs to use the network?

  (Note that the Bell Labs answer to this looks nothing like the BSD sockets API.)

• password and key management at both the system and user layer. This is especially important if encrypted filesystems start becoming pervasive. Current solutions are ad-hoc large blobs (Plan 9 had some ideas here). This interacts with things like networking, especially on shared machines; if you hand your laptop to your friend to use temporarily (with a guest account or a just-created account), you probably don't want them to be able to bring up your VPN connection to work.

• coordinating system and user choices for things like 'how do you view a web page' or 'what shows pictures'.
• can you make a general (program) configuration and options system that feels Unixy and is easily manipulated in a Unixy way, or are we doomed to things that look uncannily like the Windows registry?

  (I sure hope the answer is not 'yes'. Note that dotfiles are not the answer; they are ad-hoc single program solutions.)

• general structured interprocess communication. Many of the other problems involve programs learning about changes to the world or telling the world about changes and some of them have programs talking to other programs. They can all do this in a giant anarchy of program and problem specific mechanisms, but shouldn't there be some more general, more Unixy approach to the whole issue? Ideally one that made it easy to write little programs and shell scripts that plugged into the whole system.

• how do you process structured data in Unix pipelines and store it in 'flat' text files? Are there general mechanisms or conventions that Unix should adopt or are we doomed to a bunch of ad-hoc solutions for specific problems (such as 'a grep for json')?

• the graphical environment equivalent of the shell and shell scripts.

• what I called a good Unix glue language, something that is better than shell scripts for what shell scripts are good for while feeling right and being just as convenient. A better shell is not necessarily the right model for this.

• fair share resource allocation on multi-user or multi-use machines.

• a better permissions and restrictions model. It's become obvious that the current basic Unix model of separating and confining things by UID is not a great fit for the modern world, especially with the current set of Unix access restrictions. What is a better Unixy way to both confine processes and to give them extra permissions?

  (For example, it's absurd that you must run Apache as root in order for it to bind to port 80 and 443 and equally absurd that Apache needs to either keep root permissions or run all CGIs/etc as the same UID that is running the main web server.)

  A really well done system would allow programs to take advantage of this for handling things that they want access restricted. For example, it would be nice if the printing system didn't need its own ad-hoc private system for managing who can manipulate particular printer queues.

• integrating Unix machines in clusters and other larger network systems. My view is that none of the existing components really feel all that Unixy (NIS, LDAP, automounter, etc).

• if one of the big Unix ways of handling things is 'virtual filesystems', Unix needs to make them easier. Plan 9 argues that you can do a lot with very Unixy results but also that you need a way to make them accessible to users in order to tap their full power. What does a Plan 9 like approach look like on Unix, if it can be made to work at all? How do you integrate user-owned, user-mounted filesystems with the Unix permissions model and also not make a multi-user system explode when everyone is mounting flaky filesystems?

• /tmp on multi-user machines. This could be an entire entry by itself but /tmp is both a source of problems and a useful feature. How do you get rid of the problems while keeping the good stuff?

• what is the Unixy way to handle disk storage feature sets like ZFS and btrfs? The current approach is to shove everything from RAID on up into one huge blob, which is not really the traditional Unix way. Does Unix need some sort of fundamental shift in the underlying model of how filesystems interact with disk storage, one that will enable splitting ZFS, btrfs, etc up into nice logical multi-layered things?

  (This is more than just ZFS/btrfs. For example, I haven't seen a logical volume management system that actually feels Unixy.)

Many of these are large, abstract problems. I don't think any of them have easy or obvious answers; for some of them it's not even clear how you'd start dealing with the problem or if it's actually possible to have a 'Unixy' solution. Where current Unix systems deal with these, people often find the answers unsatisfying and non-Unixy (as I mentioned).

Unix needs to grow and a consequence of that

I hope it will not be news to people that Unix is neither perfect nor complete. By this I mean that there are things that a traditional Unix system doesn't do ideally and that can be improved as well as any number of things that it doesn't do that people really do want. I'm not talking about minor technology additions on the side (IPv6, say); I'm talking about significant additions and changes.

(If you feel that Unix is only a server OS you may feel that it is more or less complete as it stands. I disagree; my Unix is general and in many non-server environments a traditional Unix system is clearly incomplete and lacks important features. And even as a server OS only Unix is still not perfect (eg try to tell me with a straight face that SysV init is the best init system possible).)

So Unix needs to grow and change. But there is a consequence of this, one having to do with people's attitudes about changes to Unix.

To put it simply: any number of people will say that such growth is fine but only as long as the growth is (perfectly) 'Unixy', that it fits beautifully into the spirit of Unix. Unfortunately the practical consequence of saying this is that you are actually saying that Unix can't grow.

(Let us set aside for now the chances of getting a roomfull of people to agree in detail on just what the Unix spirit is and whether or not something is sufficiently Unixy.)

The reality is that it has always taken time and experimentation to work out just how to do something new in the Unixy way and the first attempts were often not very good. Even Bell Labs, the very home of the Unix way, took years and multiple attempts to get things like window systems and networking to really fit right into Unix. Other core sources of Unix development did (much) worse. Things that we think of as big parts of the correct Unix way are often relatively late additions, displacing much older and cruder solutions.

(For instance, virtual filesystems like /proc seem obviously the right way to do things but they appeared only in the late 1980s and required a whole string of previous innovations from various places. And you really don't want to know how programs like ps got information about processes before /proc.)

If you insist that people get changes to Unix correct right from the start, on the very first attempt, you are demanding that they be even more insightful about Unix than Bell Labs was. This is more or less impossible so what you are really saying is 'you can't change Unix'.

(This elaborates on two tweets of mine.)