Chris's Wiki :: blog/unix/WildcardAnnoyance

A small annoyance with Unix wildcards

December 5, 2006

Here's a small irritation with Unix wildcards: there's no generally recognized wildcard (or small set of wildcards) that matches all of the files in a directory, including dotfiles but excluding . and .., so that you could easily match all of a directory's real contents.

A plain * doesn't match dotfiles; * .* matches dotfiles, but includes . and .. too. About the best you can do is

* .??* .[^.]

However this blows up if there's no single-character dotfiles. This is a general defect in any multi-wildcard scheme, of course; the more wildcards and the more obscure they are, the more likely you are to have one that doesn't match anything.

I believe that some shells have a .* wildcard that doesn't include . and ..; on quick testing, zsh is one. (Bash and ksh seem almost able to do it but not quite, as far as I can tell.)

Interestingly, the behavior of * not matching files that start with . goes back at least as far as V5 Unix. (The V4 Unix sources have apparently been lost, and the V4 and V5 manpages are not precise enough to say whether the behavior existed in V4. If you are poking around this stuff at tuhs.org, it is helpful to know that in V6 and earlier shell globbing was done by an external program, /etc/glob; the V5 source is here.)

It's interesting to see that fairly complete shell wildcard support goes back very far in Unix history. Third Edition, released in February 1973 and the oldest Unix that tuhs.org has useful stuff for, already has *, ?, and [...] (including character ranges with -). (Since the V3 ls has a -a option to reveal dotfiles, I suspect that even V3 * wildcards didn't match dotfiles.)

Bonus trivia: the oldest ls options, present in V3, are -l, -t, -a, -s, and -d. V4 ls grew -r and -u, and after that you can go digging for yourself.

Comments on this page:

From 64.142.9.188 at 2006-12-06 18:26:09:

Bash:

$ shopt -s extglob nullglob
$ for i in .a ..b ...c .d1 ..e1 ...f1 .g. ..h. .i..
  do
     touch $i
  done
$ ls * .!(|.)
...c  ...f1  ..b  ..e1  ..h.  .a  .d1  .g.  .i..

Icarus Sparry

By cks at 2006-12-06 23:13:17:

Chalk one up in the 'things I did not know about bash' category; that's impressive. It's a pity that dotfiles apparently have to be matched by an explicit . at the start, so you can't write patterns like '+(*|.*)'.

Looking slightly deeper, 'shopt -s dotglob' does exactly this, at the expense of making * broad.

From 94.198.62.204 at 2013-07-26 10:23:27:

.* doesn’t glob . and .. in the Korn Shell.

Actually, a Korn Shell never globs . and .. period. ☺

//mirabilos (mksh developer)

Written on 05 December 2006.

«	How SSH achieves forward secrecy

Setting up switches to avoid unwanted VLAN leakage