Wandering Thoughts archives

A Bourne shell irritation: no wildcard matching operator

Here is an irritation that gets me every so often: the Bourne shell has no wildcard match operator that you can use in if checks and the like. You can do wildcard matches, but only in case statements.

(Bash has the [[ =~ ]] operator, but it uses regular expressions instead of shell wildcards. I know, I pick nits, but shell wildcards are often simpler and they match what you use in other sh contexts.)

This comes up surprisingly often, at least in the sort of shell scripts that I write. It's not insurmountable but it is inconvenient and it can make my shell scripts read less clearly. Later shells, such as Plan 9's rc, get this right and have built in wildcard matching and non-matching operators, and I have wound up using them relatively frequently.

(Yes, there is a workaround if you are doing this often enough.)

Of course, like a lot of things about the Bourne shell there are historical and philosophical reasons for this. The biggest one is a programming language design issue: you really want your wildcard matching operator to have shell support so that you do not have to keep quoting the wildcards themselves. Philosophically, the only good place to put this in the Bourne shell is as part of explicit shell syntax (ie, in a case statement); inventing a magic operator that didn't do shell wildcard expansion when used as if it was a command would be at least inconsistent.

(Tom Duff was willing to be this magical when creating rc, fortunately. It may be inconsistent but it's very convenient.)

The difficulty is compounded because the natural place to put such an operator is in test, and test started out as an external program, not something built in to the shell. If not expanding wildcards in something that looks like a command is odd in the Bourne shell, doing so for some arguments to an external program is outright serious magic.

PS: expr is not an adequate substitute for various reasons.

Sidebar: the workaround

case conditions will do variable expansion and then, if the variable expands to a wildcard, do wildcard matching on the result. So the simple way around this is to define a function:

match() {
  case "$1" in
    $2) return 0;;
  esac
  return 1
}

Then you can say 'if match $var "*.c"' and the like. If you have to you can even write vaguely crazy things like 'if match $var "*.c" && [ -f $var ];'.

Please, no automatic scrolling to the next item

As I was using Thunderbird today, I was unpleasantly reminded of a reasonably common UI idiom that I absolutely hate: having the way you page through a single 'item' (in Thunderbird, using spacebar to go through an email message) also be the way that you advance from one item to the next. The net effect is that you are paging through a message and then suddenly, surprise, you're reading the next email.

This is wrong. In the case of Thunderbird, I can explain succinctly exactly why it is wrong: when I've finished reading one message, going on to the next one is almost never what I want to do next.

Almost always, I want to either delete the message, file the message away, or reply to it. I almost never want to simply leave it sitting there in a read state. Thus, instantly advancing to the next message is 'optimizing' for the extremely uncommon action; in practice, it means that Thunderbird is doing the wrong thing almost all of the time when I hit spacebar at the end of a message.

You might wonder why I keep hitting spacebar at the end of messages. The succinct answer is that the programs are terrible at showing me when I have actually reached the end, for various reasons. That's why I want scrolling to stop and nothing to happen; it's a clear indicator that I've seen all of the message and I can now go on to dispose of it in some way.

Most programs, Thunderbird included, make then this worse with another design decision; the normal 'go back one page' key doesn't undo this, so you can't easily reverse an inadvertent advance. Sometimes it does nothing, but in Thunderbird it does something completely different and you will actually get more disoriented if you try it.

(If I used Thunderbird for more than testing our IMAP server, I think I would put together a personal Thunderbird build with this feature taken out, assuming that I could find it in the Mozilla source code. Sadly, it doesn't appear to be controllable through any configuration options.)

The only time this sort of behavior is even vaguely tolerable is when you really do faithfully simulate an endlessly scrolling display, so that people can scroll back to the previous item just as easily as they've been advanced to the next one. Even then, you should be considering carefully what people will really be doing at the end of messages. If it is not 'skipping on to the next one', maybe this is still the wrong interface.

Sidebar: the end of message problem

One of the reasons that programs are terrible at showing the end of the message is because they they don't don't allow the bottom of the content (the email text, the web page, etc) to go above the bottom of the window.

This needs some elaboration. The easiest way to see what I mean is by comparing the behavior of a GUI browser with that of a pager like less (or even lynx). When you page through something with less and the end does not naturally fall at the bottom of your screen, less advances a full page at the last page and leaves a bunch of blank space at the bottom of your terminal (in the extreme case, all but one line will be blank). When you do the same thing in a browser, the bottom of the content never goes above the bottom of the browser window; when you advance to the last page, the content scrolls by less than a full page. This means that you lack an obvious clear clue that what you are seeing is the end of the message; you have to notice that the content didn't scroll by as much as you were expecting.

Many programs should provide an easy way to change font size

Here is a thesis: any GUI program that deals with a substantial amount of text should have a browser-like set of menu options for 'increase text size', 'decrease text size', and 'reset text size'. Naturally, this should really use the same key bindings as browsers do. Browsers have shown everyone the way, so it is now well past the day that we should have to go into preferences and configurations in order to temporarily change the text size in our editors, our mail clients, our terminal windows, or whatever. Perhaps this window needs a very large font for readability on the projector; perhaps that window can be very small because it's not important. Either way, it should be easy.

(And when it is easy and convenient, it may well become common.)

Typical programs that this applies to are editors, mail clients (which are more and more browser-like in general, especially since they are being called upon to display HTML mail), and terminal windows. There are probably others, although none spring to mind right away. Anything that really is a browser should definitely be doing this, because you should be as browser-like as possible; the leading example on Gnome and KDE desktops is help viewers.

Some implementation notes:

• if the program is something where you normally only run one instance, like a mail client or a multi-window editor, I wouldn't object if it remembered the current text size when you quit and restarted it. If it does this, 'reset text size' should still reset things.

  This should definitely not happen for programs where you run a lot of instances. Just because I want this terminal window in a tiny font doesn't mean that I want all of them that way.

• it would be nice if programs had a preferences option for 'set current text size as the default text size', so that once you'd experimented to find the size of editor font (or whatever) that worked for you, you could immediately set it without trying to work out exactly what font size you'd wound up with.

• speaking of font sizes: Firefox does non-linear scaling, where 'increase font size' doesn't just add one to the current point size. Don't automatically imitate this, especially at relatively small point sizes (say, under 18 to 20 points). At the typical font sizes that terminal windows and editors and suchlike usually operate at, my experience is that I want to be able to do fine adjustments anyways because any change makes a difference.

(Firefox's large jumps in font size can actually be frustrating; my personal set of Firefox hacks makes them more fine-grained around the start point.)

Sidebar: what Firefox does for text scaling

Firefox's increase and decrease font size options multiply the base font size (whatever it is) by one of a set of constants; which constant is used depends on the level of increase or decrease in text size. The constants themselves are visible in the toolkit.zoomManager.zoomValues preference and you can see there how they are somewhat jumpy.

(In the old X days of bitmapped fonts and semi-scaled fonts, this could lead to all sorts of fun depending on your exact font. Because not all fonts were available in all sizes, Firefox used the closest size that was actually available; this meant that changing the text size could have no effect at all until suddenly it jumped significantly, as things had finally been enlarged or shrunk enough that the closest available size changed from the old font size to a new one. These days I suspect that font sizes are just rounded to the nearest point, and TrueType fonts are available in any point size you ask for (although they may not look very good at small or large ones).)

Embarrassingly, it hadn't sunk in until now that this is a user preference and so I can set and modify it without any source code changes at all. (I still have other Firefox hacks that I am carrying, but at least the number of them has shrunk over the years.)

You should have a way to purge history

Dear application programmers: I understand that you like to provide a history of recent documents, for whatever reason. But in today's environment, if you store any sort of history like this you should and in fact really must provide some way of purging that history. Ideally you would also provide a way to turn off keeping this history, too.

This does not have to be a prominent feature or element of your UI; Firefox puts their version of this in a preferences pane and a menu entry, and I don't think you necessarily need the menu entry. But it is long since time that your application had some way to do this in its GUI, since there are all sorts of reasons why users may not be comfortable having the history of their actions lying around for other people to see. Or they just may dislike seeing a pointless list of scratch files and other impedimenta in menus or the like, which is certainly the case with me and PDF viewers.

(You might think that this is an issue for browsers, not word processors or PDF viewers or the like, but not so fast. For example, your users may well have issues of business confidentiality if they do things like give their memos informative file names.)

The recent irritating example of this is evince, which wins a special bonus prize for not merely storing the full paths or URLs of many PDFs that you've viewed but also apparently re-accessing them when it converts metadata during some of its version upgrades. Hopefully I do not need to explain why randomly accessing URLs out of the blue is a very bad idea.

(Evince stores this information in .gnome2/evince/ev-metadata.xml, for those who want to do something about it. Notice how evince, unlike other Gnome programs, cleverly doesn't use gconf to store this information where a reasonably knowledgeable user might actually be able to find it.)

As an aside, my personal opinion is that if you implement such a 'recent documents' feature you should check and not list any local file that no longer exists. (Please do not check URLs to see if they still exist; see above.)

The joy of debugging other people's programs

There is a peculiar joy that is probably more common among sysadmins than programmers: the pleasure of having successfully debugged someone else's program.

Triumphing over a bug in your own program is already a pretty good feeling (assuming that it is a tricky bug, not something stupidly obvious that leaves you wanting to smack your head). Doing it to someone else's program gives you not just the reward of having squashed the bug but also the accomplishment of having sliced through enough of the program to find the problem and the place to fix it; you've triumphed over both the bug and the code.

(Also, in the sysadmin world generally you are fixing a bug not because it's there or been reported to you by someone else, but because you ran into it and it's causing you pain; fixing it gives you a direct reward.)

This joy, and the prospect of it, can be peculiarly addictive. In the stubborn pursuit of it (or simply the stubborn pursuit of irritating bugs) I've dived into all sorts of dark corners of various code bases at all hours of the day and night; sometimes successfully, sometimes unsuccessfully.

(In part I think it's addictive because understanding more and more about the code is in itself a reward. Even if you seem no closer to the bug, this increased understanding gives you that feedback you crave and it feels like progress.)

Some of the time this following has been to excess. It's an easy thing to get pulled into, especially if I feel that I'm making progress, but sometimes the right answer is to accept that I'm not going to find the bug any time soon and I need to move on somehow. This is not necessarily the right answer for a programmer, but it can be the necessary answer for a sysadmin. Still, it's not a joy that I like giving up.

(Also, I think that programmers are tacitly trained to not give up on bugs; it's part of the culture that you hunt them down sooner or later, even if they're obscure and hard to find. We pass around war stories of heroic debugging and impressively peculiar bugs, for example.)