Wandering Thoughts archives

A simple way to get a disk space usage summary

In the spirit of my little shell scripts, here's a simple way to get a useful summary of disk space usage on a Unix system:

du | sort -nr | less

(substitute the pager of your choice for less, and use 'du -x' as necessary.)

This is less pretty than the various graphical disk space visualization programs that float around, but I find it gives me pretty much the information I need. It's pretty easy to narrow down what areas of the filesystem, or what specific directories, are using up all of the space.

Applying it to various filesystems on stock machines is often interesting, especially /var; it's often a bit surprising just how much space some packages need. (For instance, a surprisingly large amount of disk space is going to /var/sadm/pkg on my Solaris test machine.)

An shell script idiom for choosing what file to operate on

This is one of those Bourne shell idioms where I show the code first and explain the effect later:

#!/bin/sh
def=$1; shift
case "$#" in
  0) if test -t 0; then
        cat -- $def
     else
         cat
     fi;;
  *) cat -- "$@";;
esac

Call this selfile. You use it in things like log scanning scripts, invoking it as:

selfile /var/log/something "$@" | ... scan stuff ...

The selfile script picks what to feed the rest of the script in the following way:

1. if you supplied any arguments, it takes them as filenames.
2. if you're invoking it interactively, it uses a default file.
3. if you're feeding something to its standard input, it just passes it on, so you can do things like 'zcat /var/log/Old/foo.* | scanner'.

These three choices turn out to be pretty much what you want in practice. You don't have to give any arguments if you want to scan today's logs, while you still have the option of feeding it stuff to digest.

The magic is 'test -t 0', which is true only if standard input is a terminal (real or pseudo); this is our somewhat crude way of telling whether we're 'interactive' or non-interactive.

(Disclaimer: while you can use this trick on standard output to have your script behave differently when it's in a pipe or otherwise having its standard output redirected, you probably don't want to do that. People get twitchy when commands produce different output like that, in part because it makes it harder to build up shell scripts bit by bit.)

A message you do not want to see from your backup software

We just got the following message today from a well regarded commercial backup system that costs a lot of money:

* host:/filesystem save: path length of 1025 too long, directory not saved: /filesystem/long/dir/path/...

Path name length limitations; just what I like to find out about in my backup software. (I suspect that the limit is 1024 characters and that the path is not exactly 1025 characters, it's just that the 1025th character triggers the error.)

I'll look on the bright side: at least it told us about it. There are worse things that can happen.

A defense of Unix that always irritates me

One of the things that some people like to trot out in defense of Unix is the old chestnut:

On Unix everything runs as a user process, so you never have blue screen of death reboots like you do with Windows; even if something goes wrong, it's just a user process and your system doesn't go down.

I like Unix, but this sort of defense of it has always irritated the heck out of me. Ignoring how it can be false in practice, it's completely disingenuous.

For a desktop user there is very little difference between losing your entire session because the X server crashed and losing your entire session because your Windows machine rebooted. That your Unix machine hasn't really rebooted, and may even be showing you a happy pleasant login screen a few seconds later, is rather cold comfort.

Unix has a lot of good things going for it. But let's be honest about them, because being disingenuous makes people twitchy even when you have an excellent case. (Which is why it irritates me.)

A little failsafe feature in 3-prong electrical plugs

I happened to notice this today: if you go look at your garden variety three prong power plugs, you'll see that the ground connector is somewhat longer than the two power connectors. Thus when you insert the plug the ground wire is connected before the power goes hot, even if things are a bit off-balance.

There's applications of this safe-before-engagement features in system administration; for example, Unix boot sequences usually configure IP filtering before the network interfaces come up, not after. You can probably think of others.

(Disclaimer: example not necessarily valid outside of North America.)

Some modest suggestions on vendor tradeshow giveaways

Part of the just finished intra-university IT conference here was a vendor tradeshow. The vendors were there to market to us, but the real purpose of going to tradeshows is of course the loot; the vendor giveaways. (And if you're hungry, the candy and snacks the vendors leave out as lures.)

In the hopes of helping vendors at future tradeshows, I thought I would share some opinions on these giveaways.

• most overused giveaway: pens. If you must give these out, please make sure they're nice to write with. (One vendor gave away the same pen they gave away last year, which was real optimism.)

• most optimistic giveaway: a frisbee. It didn't help that we're having rainy and grey weather.

• most useful giveaway: a 4-port mini USB hub (as a bonus they put it in a drawstring cloth bag). These disappeared in nothing flat (along with the USB keys they were also giving out); vendors, I see a clear unmet demand.

(We're too small for vendors to bring in that staple of the tradeshow giveaways, the t-shirt.)

Suggested future giveaways for the vendor that wants to stand out from the pack:

• if you must give out a writing implement, consider mechanical pencils; I have a lot fewer of those than pens.
• markers that are safe for writing on CD/DVD labels.
• a notebook. That's right; everyone was giving out pens, but no one was giving out something to use them on.
• a mini multitool. (This year's corkscrew & bottle opener combination was a nice try.)
• a keychain LED flashlight, small enough to be handy for shining into the dark recesses of a machine. (A vendor this year gave out a nice, startlingly bright keychain LED, but it's blue and more omnidirectional.)
• a keychain or keyring; bonus points for including a carabiner clip (please make it solid enough to feel reassuring when fiddled with).
• little mechanical puzzles. (I'm surprised that no one has tried this.)

I fiddle with things a lot (in desperation I will even fidget with pens), so bonus points are awarded for fiddleable giveaways. The people around me in technical sessions will likely award bonus points if they don't make noises when I fiddle with them.

Of course there is the ultimate giveaway, which sooner or later will get cheap enough: laser pointers. (Perhaps the thought of a hall full of sysadmins armed with laser pointers dissuades vendors.)

Safely updating Unix files

Tim Bray provides Apple some instructions on how to update files containing important data (like iCal calendars). His directions are fine at a high level, but they're actually not quite detailed enough to implement from.

So here's how you safely update Unix files:

1. create a new scratch file in the same directory as the target file (ideally mode 0600).
2. write the data to it; optionally fsync(); optionally verify the data.
3. copy the current version's permissions to the scratch file.
4. link(2) the current version of the file to the backup file name.
5. rename(2) the scratch file to the real filename.
6. fsync() your ex-scratch file, now real file, again.

(Add extra steps to taste if you want to keep more than one backup file.)

The important bit, and the important difference, is the link(2) and the rename(2). This sequence insures that if the system crashes, no matter when, you will always have some version of the file around; you will never be left with foo.old and foo.new but no foo. (This guarantee is often important when doing updating things like system binaries or /etc/passwd.)

For shell scripts, this is:

cp foo /etc/.newfile && \
  cmp -s foo /etc/.newfile && \
  ln -f /etc/foo /etc/foo.bak && \
  mv -f /etc/.newfile /etc/foo
sync

I see a lot of shell scripts that leave out the cmp. I always think that they're being awfully optimistic about cp's error handling. (Technically you should have a sync between the cp and the cmp because that's the only insurance you have against physical disk errors on the write; what cp has written has not necessarily hit the platters and thus run into any physical media problems by the time it exits successfully. Issues like this give paranoid sysadmins nervous tics.)

I have omitted the step to synchronize permissions and ownership, because I don't know of a good utility to copy them from file A to file B.

Peeking under mount points with NFS

Normally, one of NFS's irritating features is that when you mount a filesystem from a server, you don't automatically get access to any sub-filesystems mounted on that filesystem; you have to know about them and mount them yourself.

(Yes, yes, some NFS servers offer features to do this for you; such features have their own problems.)

But there's an old sysadmin trick that turns this into a feature. If you NFS mount a filesystem, such as /, you can see inside directories covered up by active mount points. If you're evil, you can write things there too. Because this is just NFS, you can do it while the system is up in regular multiuser mode without having to perturb it.

The usual situations I've wound up needing this are:

• something accidentally scribbles into a 'filesystem' that isn't mounted at the time, and it needs to be cleaned up. This usually happens either during early boot or during system maintenance when I make a mistake and don't notice it until later.
• I need to plant strategic files into the root filesystem to fake having a filesystem mounted; for example, I might need a /var/tmp that still exists before /var is mounted, or need a few device nodes to be in /dev before the dynamic /dev filesystem gets mounted.

(The other not to be discounted peculiar sysadmin use for NFS mounts is that it bypasses all of the usual rootkit infrastructure used to hide files from user-level programs. Most of those modify either user level shared libraries or system call entry points, both of which kernel NFS servers bypass.)