Wandering Thoughts archives

What typing ^D really does on Unix

Everyone knows what ^D does on Unix; it generates an end of file, so that cat or various other programs will exit (especially if they have accidentally wound up reading from standard input).

But this is just the usual effects of typing ^D; it is not what it actually does. What ^D actually does is simple but subtle:

Typing ^D causes the tty driver to immediately finish a read().

Normally doing a read() from a terminal is line-buffered inside the tty driver; your program only wakes up when the tty driver sees the newline, at which point you get back the full line. (Note that this buffering is distinct from anything that your language's IO system may be doing.)

Typing ^D causes the tty driver to stop waiting for a newline and immediately return from the read() with however much of the line has been accumulated to date. If you haven't typed anything on the line yet, there is nothing accumulated and the read() will return 0 bytes, which is conveniently the signal for end of file. If you have typed something the program will get it; because it doesn't have a trailing newline, the program's own line-buffering may take over and keep read()ing to get the rest of the line.

(Other programs will immediately process the partial line with no buffering; cat is one example of this.)

Once you've typed ^D on a partial line, that portion of the line is immutable because it's already been given to the program. Most Unixes won't let you backspace over such partial lines; effectively they become output, not input.

(Note that modern shells are not good examples of this, because they don't do line-buffered input; to support command line editing, they switch terminal input into an uninterpreted mode. So they get the raw ^D and can do whatever they want with it, and they can let you edit as much of the pending line as they want.)

Why the Unix EINTR semantics (probably) exist

Given the problems that the POSIX EINTR stuff causes to various programs, one might wonder why they exist at all. Why not make all system calls restart when signals are delivered?

What I believe it comes down to is library safety.

Suppose that your signal handler wants to change the main flow of the program. There are two plausible mechanisms for this: you can immediately transfer control back to the main flow of your program with some changes (the traditional setjmp()/longjmp() approach), or you can cause whatever your program is doing to unwind back to the point where your main code can regain control and notice the changes.

The problem with an immediate transfer of control is that if your program happened to be in a library at the time, you've preempted the library at an arbitrary time (and you probably expect to be able to keep calling the library). It's unreasonable (and in fact impossible) for libraries to be arbitrarily preemptible and re-enterable, so that it's always safe for you to jump from your signal handler to your main program and carry on as if nothing had happened.

In other words, libraries must have some way of unwinding and cleaning up their state, so that it's safe to call them again.

In a different environment, this would be an exception system. But C and Unix don't have those, so the kernel has to explicitly return control to the normal user-level code while making it visible what's happening. In other words, it has to interrupt system calls and return EINTR. At that point, a smart library can do something sensible and a dumb library is likely to return to the main program with an error, which at least lets the main program regain control and do whatever it wants.

A lot of programs don't need this, though, because they mostly or entirely use signals for temporary preemption; their signal handlers report things, or cause data saves, or whatever. For these programs, EINTR is a pain in the rear and BSD signal semantics are a blessing.

(Mind you, many of these programs are living in sin because they call library routines like fprintf() from inside signal handlers.)

Environment variables make bad switches

Someday you may be faced with a deprecation situation where you need to add a compatibility switch to your program. When in this situation it may be tempting to resort to controlling things with an environment variable, instead of giving your program yet another command line option.

(The problem with using command line options is that they add clutter, complexity, or both to your documentation, including any usage stuff that the program prints.)

Unfortunately this is not a good way to add such switches to your program, especially compatibility switches. Beyond cluttering up the environment in general, environment variables tend to leak, winding up in the environment of everything that a script runs. If any of those other programs and scripts also use your program, they're suddenly running it with different options than they expect; this may not work too well, to put it one way.

(Yes, in theory people can set the environment variable just when they run your program directly. But in practice they won't because it's too annoying; instead they'll just set the environment variable once at the start of their script and it will propagate.)

Or in less words: using environment variables makes it possible for one script to accidentally poison a different script's use of your program. Spooky action at a distance is not an asset when writing shell scripts.