Safely updating Unix files

May 13, 2006

Tim Bray provides Apple some instructions on how to update files containing important data (like iCal calendars). His directions are fine at a high level, but they're actually not quite detailed enough to implement from.

So here's how you safely update Unix files:

  1. create a new scratch file in the same directory as the target file (ideally mode 0600).
  2. write the data to it; optionally fsync(); optionally verify the data.
  3. copy the current version's permissions to the scratch file.
  4. link(2) the current version of the file to the backup file name.
  5. rename(2) the scratch file to the real filename.
  6. fsync() your ex-scratch file, now real file, again.

(Add extra steps to taste if you want to keep more than one backup file.)

The important bit, and the important difference, is the link(2) and the rename(2). This sequence insures that if the system crashes, no matter when, you will always have some version of the file around; you will never be left with foo.old and but no foo. (This guarantee is often important when doing updating things like system binaries or /etc/passwd.)

For shell scripts, this is:

cp foo /etc/.newfile && \
  cmp -s foo /etc/.newfile && \
  ln -f /etc/foo /etc/foo.bak && \
  mv -f /etc/.newfile /etc/foo

I see a lot of shell scripts that leave out the cmp. I always think that they're being awfully optimistic about cp's error handling. (Technically you should have a sync between the cp and the cmp because that's the only insurance you have against physical disk errors on the write; what cp has written has not necessarily hit the platters and thus run into any physical media problems by the time it exits successfully. Issues like this give paranoid sysadmins nervous tics.)

I have omitted the step to synchronize permissions and ownership, because I don't know of a good utility to copy them from file A to file B.

Written on 13 May 2006.
« The problem with preforking Python network servers
Weekly spam summary on May 13th, 2006 »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat May 13 01:06:33 2006
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.