Wandering Thoughts archives

A Bash test limitation and the brute force way around it

Suppose that you are writing a Bash script (specifically Bash) and that you want to get a number from a command that may fail and might also return output '0' (which is a bad value here). No problem, you say, you can write this like so:

sz=$(zpool list -Hp -o size $pool)
if [ $? -ne 0 -o "$sz" -eq 0 ]; then
   echo failure
   ....
fi

Because you are a smart person, you test that this does the right thing when it fails. Lo and behold:

$ ./scrpt badarg
./scrpt: line 3: [: : integer expression expected

At one level, this is a 'well of course'; -eq specifically requires numbers on both sides, and when the command fails it does not output a number (in fact $sz winds up empty). At another level it is very annoying, because what we want here is the common short-circuiting logical operators.

The reason we're not getting the behavior we want is that test (in the built in form in Bash) is parsing and validating its entire set of arguments before it starts determining the boolean values of the overall expression. This is not necessarily a bad idea (and test has a bunch of smart argument processing), but it is inconvenient.

(Note that Bash doesn't claim that test's -a and -o operators are short-circuiting operators. In fact the idea is relatively meaningless in the context of test, since there's relatively little to short-circuit. A quick test suggests that at least some versions of Bash check every condition, eg stat() files, even when they could skip some.)

My brute force way around this was:

if [ $? -ne 0 -o -z "$sz" ] || [ "$sz" -eq 0 ]; then
   ....
fi

After all, [ is sort of just another program, so it's perfectly valid to chain [ invocations together with the shell's actual short circuiting logical operators. This way the second [ doesn't even get run if $sz looks bad, so it can't complain about 'integer expression expected'.

(This may not be the right way to do it. I just felt like using brute force at the time.)

PS: Given that Bash emits this error message whether you like it or not, it would be nice if it had a test operator for 'this thing is actually a number'. My current check here is a bit of a hack, as it assumes zpool emits either a number or nothing.

(Updated: minor wording clarification due to reddit, because they're right, 'return 0' is the wrong way to phrase that; I knew what I meant but I can't expect other people to.)

Exceptions as aggregators of error handling

As is becoming traditional, I'll start with the tweets:

@jaqx0r: People complaining about "if err != nil" density in #golang don't check errors in other languages. Discuss.

@thatcks: In exception-based languages, you often get to aggregate a bunch of similar/same error handling together in one code bit.

As usual on Twitter, this is drastically crunched down to fit into 140 characters and thus not necessarily all that clear. To make it clear, I'll start with a really simplistic example of error checking in a routine in Go:

func (c *MyConn) reply(code int, msg string) error {
   emsg, e := utils.encode(msg)
   if e != nil {
      return e
   }
   e = c.writecode(code)
   if e != nil {
      return e
   }
   e = c.writemsg(emsg)
   if e != nil {
      return e
   }
   e = c.logger.logreply(code, msg)
   return e
}

If it runs into an error, all that this code does is abort further processing and returns the error to the caller. There is a slightly less verbose way to write this but regardless of how you write it, you have to explicitly check errors each time even though you're handling them all the same way.

In an exception-based language like Python we can write the code so that all of these 'if error, abort' checks are aggregated together. In a slightly more elaborate example we might write something like:

def reply(self, code, msg):
   try:
      emsg = utils.encode(msg)
      self.writecode(code)
      self.writemsg(emsg)
      self.logger.logreply(code, msg)
   except (A, B, C) as exp:
      ... cleanup ...
      raise MyException(exp)

Since the action taken on errors is the same for all cases, we've been able to aggregate it together into a single exception handling block with a single set of code. This clearly results in less code here.

The more that your code wants to do basically the same thing on errors (perhaps with some variations in messages or the like), the more you will benefit from error aggregation through exceptions and the more onerous and annoying it is to repeat yourself every single time. Conversely, the more different your error handling is, the less you benefit; in fact in some environments the exception-based version can wind up more verbose. My perception is that often you have essentially the same handling of a lot of errors and so the aggregation offered by exceptions will wind up saving you code and repetition.

(There are tricks to avoid repeating yourself with explicit error checking but at least some of them wind up distorting the flow of your code, for example by inserting an intermediary function whose only real purpose is to aggregate the error handling actions into one place.)

So, I think that people who complain about the density of explicit error checks in Go (and similar languages) have a real case here. I don't think Go will ever solve it, but these people are not necessarily writing sloppy code in other languages, just efficiently structured code.

(This is basically what I wrote back in Exceptions as efficient programming.)

Sidebar: Some pragmatic arguments for the Go way

There are arguments about code readability here, of course. The Go version makes it explicit that the only action done on error each time is to return immediately from the function, while the Python version moves that to a separate block of code. It's not very distant in this contrived version, but you can imagine a more realistic one where there was a substantial amount of code that was being wrapped inside a single exception handler (thus moving the two much further apart). As a structural matter, the Go example also clearly handles all error cases that can arise from each function call, while in the Python one we've assumed that we know all of the exceptions that can be raised underneath us.

Go's tradeoffs here strike me as the correct ones given Go's overall goals, even if I'm not sure I like them. Still, I can't say that handling explicit errors has been any particular pain in my Go code to date.

My current views on Rust (the programming language)

Let's start with my tweet:

Modern languages like Rust and Haskell often give me the feeling that I am not smart enough and determined enough to program in them.

In the geeky programmer circles of the Internet that I fish for information in, Rust is very much the latest hot thing. It's generally presented as a better C/C++ that is (or should be) just as fast while being far more memory safe. All of this sounds very nice; who wouldn't want a C with better memory safety, after all?

My problem with Rust is that, as far as I can see from reading about it in passing, it is a fairly complicated language (due in large part to its determination to be a memory safe systems programming language). To read about Rust code is to be immersed in mutable and immutable variables, transferable references, borrows, macros, apparently complex types, and many other things. I'm probably smart enough to code in Rust if I was dedicated enough to learn all of the things that you need to know to write and read Rust code fluently. In practice I'm not.

One reason I'm not that dedicated is that it turns out I no longer have a use for 'like C but with better memory safety' (and a modern type and macro system, I think). And this is because I no longer write 'systems programs', broadly construed. If I just need a program, I can write it in Go or Python. If I need a program that goes fast or uses relatively little resources, I'll write it in Go. I write C programs these days only in two circumstances: if I need an absolutely minimal sized executable, or if I need a program that is right down to the metal of Unix system calls.

(One use of 'down to the metal' programming is benchmarking and testing, where I want to be sure exactly what the program is asking the kernel to do.)

But that's the sober rational reason. The emotional reason and my gut reaction to Rust is that it's too complicated to be appealing to me. More than that, it's complicated about things I no longer give a damn about, such as memory management without garbage collection. Rust is a language for smart people who care very deeply about certain issues, and I do not.

(I can completely see why Mozilla wants Rust and would not consider Go an adequate substitute. But I am never going to write a browser.)

And despite everything I've written here, there's still a part of me that wants to like Rust because damnit, it's cool and in theory it is right up my 'systems programming' alley and any number of smart people think it's a great thing. So I keep reading Rust news every so often (in much the same way I keep reading Haskell news). No one ever said I was a completely sensible person.