Wandering Thoughts archives

2016-09-16

A shell thing: globbing operators versus expansion operators

If you've been using a Unix shell for long, you may be familiar with the '[...]' wildcard, which can be used to match a character range (or a bunch of characters):

ls -lt logfile.[1-5].gz

If you've used Bash or a number of other shells for a while, you may also be familiar with '{..,...}':

touch afile.{one,two,three}

There is an inconvenient chasm here between these two very similar things. Wait, a chasm? Sure. Imagine that you want to create afile.1 through afile.5. Can you write this in a nice compact way as the following?

touch afile.[1-5]

The answer is no, and this is the chasm in action. You can use '[1-5]' to match logfile.1.gz through logfile.5.gz, but you can't use it to generate 1 through 5 for touch. Similarly, you can't use {...} as part of a wildcard match, eg:

ls -lt afile.{c,h,go,cpp,py,rb,el}

What is happening here is that modern shells have two sorts of operators, wildcard globbing operators and expansion operators. Expansion operators are simply text substitution and expansion, so 'x.{a,b,c}' expands out to 'x.a x.b x.c' regardless of what files currently exist. Wildcard globbing operators match filenames and only expands out to filenames that match; if nothing at all matches, it's either an error or the operator produces itself as literal text.

(In other words, if you do 'touch nosuchfile.*', you get a file called 'nosuchfile.*'. The operator producing itself is the standard behavior but some shells have an option to make a failed glob into an error.)

The chasm between the two fundamentally exists because the shell can't read your mind about what you want. To return to my earlier example, if you write:

touch afile.[1-5]

and you already have a file called afile.1, do you actually want to update its timestamp and do nothing else, or you want to create afile.2 through afile.5 as well? The shell can't tell, so it must pick one or the other. It is this decision that creates the distinction between wildcard globbing operators and expansion operators.

(Globbing came first, by the way. Expansion operators got added later, although in the end the Bell Labs people decided that having a '{...}' feature was sufficiently useful that an equivalent was included in Tom Duff's rc.)

(This entry was sparked by Advancing in the Bash Shell, via John Arundel, which got me thinking about how '{..}' is kind of weird in the shell.)

unix/ShellGlobVsExpansion written at 01:05:15; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.