Wandering Thoughts archives

An Amanda gotcha with dumps to disk

One of the configurations that Amanda supports these days is dumping to disk instead of to tape. The leading way to do this is to use your disk space as virtual tapes, using the 'file' output driver (per here).

When you do this, it is vital to remember that to Amanda, these are tapes. Very fast tapes, but tapes. And they have all of the limitations of tape. Such as that only one thing can be written to them at once.

So if you do not set up a holding disk in addition to your disk space for the 'tapes' (because why would you need a holding disk when you're dumping to disk anyways), Amanda will behave just like it would with a real tape: it will serialize all your backups, because they have to go to directly to tape.

(It is surprisingly hard to notice this, because the daily Amanda reports do not mention that they are dumping directly to tape and does not have the start time of the various dumps, which would have let us see this clearly. However, amstatus will tell you that only one dumper was ever used, which is a dead giveaway, and you can run amstatus even after amdump finishes.)

Why have an MX record to yourself?

In a recent entry, Russell Coker brings up an issue:

One issue that has been the topic of some pointless discussion is whether a mail server should have an A record or an MX record.

It's harmless not to have an MX record that points to yourself, but having one can save people a DNS query in many situations.

Answers to DNS queries have three sections: answer records, authority records, and additional records. Authority records are the NS records of the authoritative nameservers (and SOA records for negative answers); additional records are A records for any NS or MX records in the rest of the answer.

So if you have a self-pointing MX, anyone who queries your authoritative nameservers will get your MX record and your A record in one query. If you don't have an MX record, they will have to make two queries; one to find out that you don't have an MX record, and the second to get your A record.

(Similar clever tricks can be pulled through NS records. For example, if you make your web server one of your nameservers, people who go to your website will probably save a DNS lookup. But there are downsides to such tricks.)

There are two flies in the ointment:

• nameservers only return additional records that they know at the time; if a caching nameserver has discarded your A record but not your MX record, that's it.

• some caching nameservers, including at least djbdns's dnscache, deliberately don't include authority records or additional records in their replies in order to make their replies smaller.

A trivial script: wcat

Here's wcat, which is short for 'web page cat':

#!/bin/sh
exec wget -q -O - "$@"

(At some point I may need to add '-e robots=off' to this, but I haven't needed it yet.)

I have a flaw when it comes to my little scripts: often, I think that sometime is too small and trivial to bother turning into a script, so I don't for ages. And then when I give in and make the script, I rediscover that automation promotes action, because I start using the new script all over.

This is exactly what's happened with wcat; now that I have it, I've been discovering all sorts of situations where it is just what I want, and I can't imagine how and why I lived without it for so long.

(I suspect that pretty soon I am going to find myself writing wless, which will just be something like 'wcat "$@" | less'.)

A pyramid of little scripts: nsdig

One of my DNS diagnostic tools is a program called nsdig; it makes a query to all of the authoritative servers for a zone. This makes it convenient to both look for inconsistencies and see what the authoritative servers are saying, as opposed to whatever is filtering through your local caching servers.

As another one of my little shell scripts, and as a practical example of how little utilities stack on top of each other to everyone's benefit, here's nsdig and its component bits:

; cat nsdig
#!/bin/sh
case "$#" in
   3) nssrc=$3;;
   2) nssrc=$2;;
   *) echo usage: nsdig T WHAT [DOM] 1>&2
      exit 1;;
 esac
 for i in `nsaddrs $nssrc`; do
     echo "---- $i ----"
     dig +norecurse $1 $2 @$i
 done

(The third argument is necessary for cases like 'nsdig a www.foo.com foo.com'.)

; cat nsaddrs
#!/bin/sh
addr `sdig ns $1` | sort -u

; cat addr
#!/bin/sh
for i in "$@"; do
  sdig a $i
done

; cat sdig
#!/bin/sh
exec dig +short "$@"

(dig is the standard DNS lookup utility. Honesty compels me to admit that the version of addr that I actually use is a much more complicated Python program with various additional features that are irrelevant for nsaddr's usage.)

The one building block that is not as useful as it could be is dig, which sometimes insists on sprinkling its output with extraneous bits. (For example, even the short dig output for TXT queries has quotes around the results, which are almost always surplus.)

The advantage of vendor packages

The advantage of using vendor packages is that someone else worries about taking care of program <X>, tracking upstream releases and bugfixes and security issues, making sure that it works, and (with a decent package management system) mostly handling upgrades from version Y to version Z.

This may seem petty, since after all it's easy enough to keep track of something like Apache. The problem is that without vendor packages, it's not just Apache; it's Apache and GNU Emacs and LaTeX and GCC and Perl and Python and so on without end. Tracking things by hand doesn't scale, especially in an environment with other demands on your time, and pretty soon your local software is quietly rotting.

(By 'vendor' I really mean 'anyone who handles package maintenance for me'; as things like blastwave.org and Fedora Extras demonstrate, it need not be the actual OS vendor themselves.)

This is why the Solaris ssh issues bug the heck out of me; Sun is falling down on its end of this, removing the benefit of them handling ssh for me and replacing it with a big hassle. This is one of the areas where doing a half-assed job is worse than not doing the job at all, because it lets people down partway through.