Why I use exec in my shell scripts

January 24, 2012

As with the little example yesterday, a fair number of my shell scripts end with running a program and when they do, I almost invariably go the little extra distance and do it with exec. In the old days, the reason to do this was that it used slightly less resources, since it got rid of the shell process and left only the process for the real program you wound up running. But, while I was around then, the reason I use it today isn't that; it's that it lets you freely edit the script while that final program is running.

At this point some of you may be going 'wait, what?' That's because most Bourne shell implementations are a little bit peculiar.

In most interpreted languages on Unix (like Python, Ruby, and Perl), the interpreter completely loads and parses the script file before it starts running it. This means that once your script has actually started running, once that initial load and parse has finished, you can freely change the script's file without the interpreter caring; it will only look at the actual file and its contents again if and when you re-run your script.

Bourne shell implementations have historically not worked this way (and it's possible that it's actually impossible to preparse Bourne shell scripts for some reason). Instead they not only parse the script on the fly as it executes, but also they read the file on the fly as the script runs. This means that if you edit a shell script while it's running you can literally shuffle the code around underneath the script. When the shell resumes reading and parsing the script after the current command finishes, it can be reading from partway through a line, from something that it had already read, or (if you deleted text) wind up skipping over something that it should have run. This often causes the shell script to fail with weird errors or, worse, to malfunction spectacularly. This can happen even if the shell is on the last line of the script.

But if you end a shell script with exec, you avoid this. The actual shell interpreter effectively exits (by turning itself into the actual program) and so there's nothing there to try to read anything more and get confused by your edits.

(Of course nothing helps if you can't use exec; then you just have to remember to never edit the script while it's running, at least with an editor that overwrites the file in place.)

Sidebar: a detailed example of what happens

Let's start with a little script:

echo "a"

Run this script. While Firefox is running, edit it so that the echo string is four or five characters longer (using vi or some other editor that overwrites files in place). When you exit Firefox, the script will complain something like 'script: line 4: efox: command not found'.

When the shell was running Firefox, its read position in the file was just after the newline at the end of firefox. When you edited the script and added more letters, that same byte position was now pointing to the e in the 'firefox'. When Firefox exited and the shell resumed reading from that byte position, it read 'efox<newline>', saw a perfectly valid command execution, and tried to run 'efox' (and failed).

(It reports that this happened on line 4 because it knew it had already read three lines, so clearly this is line 4. As a corollary, you can't trust the line numbers that are printed when something like this happens.)

Comments on this page:

From at 2012-01-24 02:23:05:

Don't most editors write to another file and make an atomic rename?

By cks at 2012-01-24 11:21:31:

Most of the editors that I use don't. Vi doesn't by default, for example, and it's a popular editor for the people who write scripts.

From at 2012-01-29 21:27:43:

This is the simpler/easier fix in .vimrc:

set backup " xemacs like file backup

Written on 24 January 2012.
« Every so often, I solve a problem with a hammer
The death of system administration: I'm all for it »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Jan 24 00:06:12 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.