Wandering Thoughts archives

A gotcha with the format of dump archives

Most archive formats, such as tarballs, cpio archives, and zip archives, interleave the file names (and their directory structure) with the file contents. The dump filesystem backup program such as Solaris ufsdump do not. The dump archive format starts with an index that has all of the filenames and the directory structure; after this comes the actual file content, labeled by inode number.

This has an important consequence for detecting damaged archives. In an interleaved format like a tarfile, getting a full file listing requires reading the entire archive and thus checks that it's intact. However, in dump's format getting a full file listing only requires reading the first bit of the archive; it does not guarantee that anything past the index is uncorrupted or that all of the listed files are actually present in the archive.

There are two ways of verifying dump archives; one is general and the other requires the Linux version of restore.

• Linux's restore has a -N flag that causes it to not write anything to disk, so you can do a 'dryrun' restore that reads the entire archive with something like restore -x -o -N -f ....

  Linux's restore can generally read the output of dump from non-Linux systems. In particular, we have tested it with Solaris 8 ufsdump and it works fine.

• you can use a full file index to work out the highest inode number in the dump archive and then restore just it. Because dump usually writes out files by increasing inode number, this generally forces the restore program to read the entire archive.

  (However, I don't know if this will detect missing files, things that are in the index but not in the file contents.)

If neither of these are workable or good enough, your only option is to do a full restore to a scratch partition somewhere.

(For reference, the home of the Linux dump and restore is here. While dump requires ext2 specific header files and libraries, I believe that restore can be compiled on most any system with some work.)

The scope of shell history

One of the little divides in Unix is the one between people who set up their shell to have a per-shell command history (a local history) and the people who set their shell up to use a global history.

While I know perfectly sane and sensible people who are in the global history camp, I am firmly on the local history side, because to me my shell history is contextual. When I hit cursor up or tell a shell to redo the last command, I want it to do redo the command that is right there in that window; I do not want it to redo the last command I did anywhere. (In fact, I doubt I could keep track of the last command I did.)

I suspect that a lot of sysadmins will fall into the local history side. Sysadmins operate in so many different contexts, some of which are split apart by necessity and cannot be joined, so it's very hard to have a truly global history that covers every command, on every machine, as every user. My guess is that once people start working with only semi-global history that they will prefer to go whole hog to local history.

Although I don't know if any shell supports it, there is an intermediate option: maintain both a local history and a common global history, but only look at the global history if the local history is exhausted. That way you keep local context but also get to pull back that neat ad-hoc pipeline you came up with two days ago in a window that you have long since closed down.

(Before you tell me about it, this is not quite what bash does; it freezes the 'global' history the moment you start it. Bash uses its own hybrid model that makes my head hurt.)