Wandering Thoughts archives

How I shot my foot because the Bourne shell is different

I have an invariable, reflexive habit in the Bourne shell, which is that I call my for loop variables $i. This reflex is so ingrained that if I try to fight it, I can wind up writing loops that look like this:

for avar in ....; do
    somecommand $i
done

I may have carefully written the for loop using a sensible, non-$i variable name, but then when I was writing the body of the loop I forgot and reflexively used $i. This is always a fun, forehead-slapping issue to debug (even if it often doesn't take long, especially in tiny for loops).

(Much of this isn't unique to the Bourne shell; like any number of people, I normally use i as my default loop variable name no matter what language I'm working in.)

Recently I wrote a script where the top level looked something like this:

... various functions ...
reporton() { ... }

for i in $@; do
    case "$i" in
       magic) reporton "magic $i" $(magic-thing $i) ;;
       ...) ... ;;
       /cs/htuser/*) reporton "$i" $(webdir-fs $i) ;;
       ...) ... ;;
       *) # assume username
          reporton "<$i>" $(user-fs $i)
          reporton "<$i> other" $(webdir-fs /cs/htuser/$i) ;;
    esac
 done

Most of the various arguments I could give the script worked fine. In the username case, the first reporton worked properly but then the second one failed mysteriously, printing out weird messages. To make it more puzzling, the same reporton worked fine when run independently (in the /cs/htuser/* case).

It took a bit of time before the penny dropped. Several of my shell functions had their own for loops, and of course I had reflexively written them using $i as the loop variable. Since I was writing this in more or less pure Bourne shell, I wasn't using 'local i' in any of these functions and so everyone's for loops were changing the same global $i variable.

For most of the functions, this worked out; they didn't call other $i-changing functions inside their own for loops, so the value of $i was stable in their for loop bodies. But at the top level this wasn't the case; I was obviously calling the whole stack of functions and was having $i's value changed out from underneath me. Most of the time this didn't matter because I only used $i once (before its value got changed by the functions I called). The 'assume username' case was the exception; as you can see, I ran two reporton's in succession and used $i with each. When I got to the second reporton, $i's value was completely mangled and things blew up.

Most languages that I deal with are lexically scoped languages, where reusing the same name for variables in different scopes just works (in that each version of the variable name is completely independent). Lexical scoping is so pervasive in my programming languages that I think of it as the normal, default case. The Bourne shell is one of the few exceptions; it's dynamically scoped, and so all of the $i's are the same variable and my various usages of $i were stepping on each other. Since it's the rare exception and I don't do complicated Bourne shell programming very often, I completely forgot this difference when I wrote my script. Hopefully I'll now remember it for the next time I write something in the Bourne shell that's sufficiently complicated that it uses functions and multiple for loops.

(Awk is another language that I deal with that normally has dynamic scope, but I've only ever written a few pieces of awk that were complicated enough to use functions (such as this one).)