A Bash test limitation and the brute force way around it

June 23, 2015

Suppose that you are writing a Bash script (specifically Bash) and that you want to get a number from a command that may fail and might also return output '0' (which is a bad value here). No problem, you say, you can write this like so:

sz=$(zpool list -Hp -o size $pool)
if [ $? -ne 0 -o "$sz" -eq 0 ]; then
   echo failure
   ....
fi

Because you are a smart person, you test that this does the right thing when it fails. Lo and behold:

$ ./scrpt badarg
./scrpt: line 3: [: : integer expression expected

At one level, this is a 'well of course'; -eq specifically requires numbers on both sides, and when the command fails it does not output a number (in fact $sz winds up empty). At another level it is very annoying, because what we want here is the common short-circuiting logical operators.

The reason we're not getting the behavior we want is that test (in the built in form in Bash) is parsing and validating its entire set of arguments before it starts determining the boolean values of the overall expression. This is not necessarily a bad idea (and test has a bunch of smart argument processing), but it is inconvenient.

(Note that Bash doesn't claim that test's -a and -o operators are short-circuiting operators. In fact the idea is relatively meaningless in the context of test, since there's relatively little to short-circuit. A quick test suggests that at least some versions of Bash check every condition, eg stat() files, even when they could skip some.)

My brute force way around this was:

if [ $? -ne 0 -o -z "$sz" ] || [ "$sz" -eq 0 ]; then
   ....
fi

After all, [ is sort of just another program, so it's perfectly valid to chain [ invocations together with the shell's actual short circuiting logical operators. This way the second [ doesn't even get run if $sz looks bad, so it can't complain about 'integer expression expected'.

(This may not be the right way to do it. I just felt like using brute force at the time.)

PS: Given that Bash emits this error message whether you like it or not, it would be nice if it had a test operator for 'this thing is actually a number'. My current check here is a bit of a hack, as it assumes zpool emits either a number or nothing.

(Updated: minor wording clarification due to reddit, because they're right, 'return 0' is the wrong way to phrase that; I knew what I meant but I can't expect other people to.)


Comments on this page:

By mtk@acm.org at 2015-06-23 07:35:29:

what is your objection to using bash's 'boolean' syntax where you can use && and || and get short-circuited evaluation?

If you are specifically in Bash (i.e. there is no requirement of being sh-compliant), why not use Bash-isms? A '[[' test operator is more flexible than traditional '[', and it does short cirquiting.

Try this on for size:

sz=$(zpool list -Hp -o size $pool) && (( sz > 0 )) || { echo failure ; }
By cks at 2015-06-23 16:47:16:

In this case the actual failure handling is multiple things (and involves a continue). This might well work in a '|| { ...}' thing, but I much prefer to write it as an explicit if block so it reads clearly.

Now that I've looked at them, I'm waffling on using Bash's '[[' and short circuiting ||. On the one hand they solve this directly. On the other hand, I don't think my co-workers are familiar with them, so I'd be putting a roadblock in the way of them easily reading this script. On the third hand, they're probably not likely to anyways.

(The script is specifically in Bash not because it needs Bashism but for what turn out to be complex historical reasons that are interesting in their own right. We probably want to keep doing it, though; the /bin/sh on the current machines has some oddities of its own.)

Since you're explicitly using bash, how about

 if [ $? -ne 0 -o ${sz:-0} -eq 0 ]

"Default Value" expansion: evals as 0 if $sz is unset or null.

By dozzie at 2015-06-24 07:53:20:

@mtk:

what is your objection to using bash's 'boolean' syntax where you can use && and || and get short-circuited evaluation?

Bash has bad history of changing subtle details of [[ ]] syntax between minor releases. This leads to easily introduced bugs that are very difficult to track, while the gain of using [[ ]] over POSIX/SUS-compliant syntax is typically negligible.

By cks at 2015-06-24 15:43:19:

@S: Nice! I hadn't even thought of the ${sz:-0} approach; that's clever.

(Probably too clever for a script that I want my co-workers to read, sadly.)

Written on 23 June 2015.
« Modern *BSDs have a much better init system than I was expecting
I've finally turned SELinux fully off even on my laptop »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Jun 23 00:44:27 2015
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.