Chris's Wiki :: blog/programming/ShellScriptVerbosity

My common patterns in shell script verbosity (for sysadmin programs)

October 27, 2019

As a system administrator, I wind up writing a fair number of scripts that exist to automate or encapsulate some underlying command or set of commands. This is the pattern of shell scripts as wrapper scripts or driver scripts, where you could issue the real commands by hand but it's too annoying (or too open to mistakes). In this sort of script, I've wound up generally wanting one of three different modes for verbosity; let's call them 'quiet', 'dryrun', and 'verbose' In 'verbose' mode the script runs the underlying commands and reports what exactly it's running, in 'quiet' mode the script runs the underlying commands but doesn't report them, and in 'dryrun' mode the script reports the commands but doesn't run them.

Unfortunately this three-way setup is surprisingly hard to implement in a non-annoying way in Bourne shell scripts. Now that I've overcome some superstition I wind up writing something that looks like this:

run() {
  [ "$verb" != 0 ] && echo "+ $*"
  [ "$DOIT" = y ] && "$@"
}

[...]

run prog1 some arguments
run prog2 other arguments
[...]

This works for simple commands, but it doesn't work for pipelines and especially for redirections (except sometimes redirection of standard input). It's also somewhat misleading about the actual arguments if you have arguments with spaces in them; if I think I'm likely to, I need a more complicated thing than just 'echo'.

For those sort of more complicated commands, I usually wind up having to do some variant of this code as an inline snippet of shell code, often writing the verbose report of what will be run a little bit differently than what will actually get run to be clear about what's going on. The problem with this is not just the duplication; it's also the possibility of errors creeping into any particular version of the snippet. But, unfortunately, I have yet to come up with a better solution in general.

One hacky workaround for all of this is to make the shell script generate and print out the commands instead of actually trying to run them. This delegates all of the choices to the person running the script; if they just want to see the commands that would be run, they run the script, while if they actually want to run the commands they feed the script's output into either 'sh -e' or 'sh -ex' depending on whether they want verbosity. However, this only really works well if there's no conditional logic that needs to be checked while the commands are running. The moment the generated commands need to include 'if' checks and so on, things will get complicated and harder to follow.

Comments on this page:

By dozzie at 2019-10-27 06:14:51:

The problem with this is not just the duplication; it's also the possibility of errors creeping into any particular version of the snippet. But, unfortunately, I have yet to come up with a better solution in general.

I usually do something like this:
 set_routing() {
   if [ -n "$VERBOSE" -o -n "$DRY_RUN" ]; then
     echo "ip route add ..."
   fi
   if [ -z "$DRY_RUN" ]; then
     ip route add ...
   fi
 }
When you wrap all your change operations in functions, you add the --dry-run support to those functions, and then the script executes its logic in all cases the same way. Sort of opposite strategy to your run function.

By Tony Finch at 2019-10-27 10:55:12:

I use set -x for verbose mode. Ansible has eliminated most of my need for writing my own do-nothing test mode.

I should use make more, because that would give me verbose mode and do-nothing mode and parallel mode for free.

By Ewen McNeill at 2019-10-27 19:27:03:

Like Tony, I tend to use “set -x” for verbose, and Ansible or similar for anything complex that needs to be idempotent. I’ve also used “make” in the past where the problem naturally divides into “do the appropriate subset of the steps depending on what changed” and it’s all local to a single system. For the right kind of dependency problem “make” works very well.

I do also use code generation — generating a shell script that is run by piping to “sh” or “sh -x” or similar — fairly frequently. It is possible to generate a script with its own conditionals in it, but that requires care around escaping. Often when the script generator needs to be more complex like that, I’ll write the generator in another language like Python. (Mostly I don’t have a “dry run” mode for anything that isn’t “generate a shell script”, or written in Ansible or similar — but I constantly use the the dry run/check modes in Ansible, Salt, etc. I guess if I feel I need a dry run option it already feels complex enough to need more assistance than just Bourne shell :-) )

Ewen

By Aristotle Pagaltzis at 2019-10-28 20:21:45:

The main subject is a tricky one that I have no good solution for, but what I can offer is this:

I need a more complicated thing than just 'echo'.

If you can require bash, that thing is just
printf '%q ' "$@" ; printf '\n'
Or if you can’t stomach the minor imperfection in its output and/or you can stomach the trickery of the following, you can also go with
printf "${*//*/%q}\n" "$@"

Written on 27 October 2019.

«	An incorrect superstition about running commands in the Bourne shell

An interesting little glitch in how Firefox sometimes handles updates to addons