Wandering Thoughts archives

How to do locking in shell scripts

To do simple locking in shell scripts, you need an atomic system call that fails if the target already exists, and that is directly exposed in a utility program. Ignoring for a moment what System V did to ln, there are two good candidates on modern Unix systems: link(2) and mkdir(2).

Mkdir is actually the better of the two (because you don't have to fiddle around with temporary files), but ln is the more common one, probably for two reasons:

1. with mkdir you can't atomically associate some extra information, like a PID, with the lock. You can put a PID file inside the directory, but there will be a time when it doesn't exist although you have the lock.
2. there was no mkdir(2) system call in V7; it was only added in UCB BSD, and didn't make it into System V until fairly late. So for a fair time mkdir wasn't actually suitable for this, and everyone got into the habit of using ln.

General locking with mkdir is simple:

mkdir LOCK 2>/dev/null || exit 0
echo running locked
rmdir LOCK

Locking with ln (ignoring System V) is somewhat more complicated:

echo $$ >$$
if ln $$ LOCK 2>/dev/null; then
  echo running locked
  rm $$ LOCK
else
  rm $$
fi

In both cases you really want a private directory for the locks. If you have to use a shared directory like /tmp, I believe the ln approach is the only one that can stand up to hostile attacks (provided that you use something like mktemp to securely create the temporary file).

What System V did to the poor ln command

On V7, UCB BSD 4.x, SunOS 2.x, 3.x and 4.x, the *BSDs, Linux, and anything using GNU fileutils:

; touch a b
; ln a b && echo oops
ln: b: File exists

On System V, including Solaris 8:

; touch a b
; ln a b && echo oops
oops

That's right: the System V version of ln goes to significant extra work over just calling link(2), and in the process is so unhelpful that people had to make a new utility called link to do the right thing. (Fortunately, this behavior is no longer SuS compliant, per the ln spec.)

This is one of those things that sticks in my mind, because I had the distinction of being the first person to try to run CNews on a System V machine. CNews has a lot of shell scripts and had been developed on V7 and BSD systems, so all of the shell scripts did their locking with 'ln temp LOCK'; the consequences of the locking not actually working were reasonably spectacular and quite memorable. Fortunately the machine wasn't being used by anyone else at the time.

(The authors of CNews were as appalled as I was, and promptly introduced a little newslock program and switched the shell scripts to using it, so they had behavior they could count on.)

I was reminded of this mess recently when I wrote a shell script that needed to both do locking and run on our Solaris 8 systems. I was reasonably displeased to discover that Solaris 8 still has this bit of braindamage.