Wandering Thoughts archives

A Unix irritation: pipeline status

Let's start with a Bourne shell irritation. Suppose you have a command sequence that looks like:

may-fail | grep ...

Further suppose that you want to know whether may-fail actually failed. In the normal Bourne shell, you're out of luck; the exit status of a pipeline is the exit status of the last command, and grep will probably succeed no matter what happens to may-fail. You're going to need to use a temporary file instead of the pipeline or get very creative.

It's tempting to blame the Bourne shell for this, but I think that its hands are at least partially forced by our old friend SIGPIPE. Consider this pipeline:

produce-lots | sed 10q

What's the exit status of this pipeline, assuming that produce-lots would succeed if left alone?

Sed will exit successfully after writing ten lines. But this closes the pipe, which means that produce-lots gets either a SIGPIPE signal or an EPIPE write error (if SIGPIPE is being ignored). Unless produce-lots is specially coded, it is going to exit with some sort of error status. If the Bourne shell only considered pipelines successful if all of their commands succeeded, this sort of pipeline would fail, much to people's surprise.

(The shell could consider pipeline commands that exited on SIGPIPE to have succeeded, but that's an unreliable hack.)

Hence, the only reliable exit status is the exit status of the last command in the pipeline; everything else can wind up 'failing' when they have not in fact failed, it's just that the pipeline has shut down early.

The NFS 'reserved ports' option and why you care

Various NFS servers have various different names for an option to, well, I can do worse than quote the Linux exports(5) manpage:

This option requires that [NFS] requests originate on an Internet port less than IPPORT_RESERVED (1024).

On traditional Unix systems, so called 'reserved ports' can only be used by root. (On non-Unix systems there's often no restriction, which has periodically caused bad Unix protocols some heartburn.)

Why does this option exist, and why does it matter?

Suppose that you're an attacker with a normal Unix login on an NFS client, and the NFS server does not require reserved ports. This means that you can talk to the NFS server yourself with ordinary UDP (or TCP) packets; the client machine is already authorized to talk NFS to the server (at least for filesystems exported to it), and the server isn't noticing the difference between you and the machine's real kernel-based NFS mounts.

This is disastrous. Classic NFS servers trust the UID information that the NFS client sends them (this is the difference between NFS and real network filesystems), and you are in full control of that information; you can make whatever NFS request you want as whatever UID you want, and read the server replies. Congratulations, you can now do anything over NFS that any UID on your client machine can do.

(It's as if you'd compromised the NFS client but without the hassle of actually having to get root.)

The NFS server insisting that NFS requests come from reserved ports on the client machine stops this; as an ordinary user, you can't send packets from reserved ports, and when you send packets from non-reserved ports the NFS server will just ignore them.

(This entry is brought to you by Solaris, the operating system with defaults from 1996 (if not earlier). Turn on nfssrv:nfs_portmon on your Solaris 10 machines today.)

Sidebar: the one technical barrier

In order to talk to an NFS server, you need a valid NFS filehandle, ideally for some directory (because given a directory filehandle, you can walk back through .. to get access to all of the filesystem). There are two ways to get one (ignoring network snooping).

First, you can just ask the server for one via the NFS mount protocol. If the NFS server isn't checking for reserved ports, the server's mountd may not be either, and your machine already has the required permissions in general.

Second, you may be able to make one up by hand. Various NFS servers form NFS filehandles in predictable ways and you can usually find out all or nearly all of the information you need in order to duplicate what the server does. Even if you can't get all of the information you can just start guessing and querying the server to see if you've invented a valid filehandle; typical servers do not log 'invalid filehandle' errors anywhere, because their logs would explode.

Sometimes brute force is the answer (on Unix)

Suppose that you want to extract all of the IP addresses used by your (OpenBSD) firewall's PF rules, so that you can (for example) validate that all of the internal addresses mentioned still exist.

(For the purposes of this exercise we can ignore IP addresses in PF tables, because those are already easy to dump. Also, I am ignoring IP address ranges, netblocks, and so on; I am just interested in specific IP addresses.)

If you think that parsing PF's pf.conf file yourself is the answer to this problem, you have two problems (to riff from a famous remark). That way lies rewriting PF's rule parser yourself, and the language is not a simple one. You could parse the output of 'pfctl -s rules', which dumps the post-parsed rules in a simplified, expanded form that looks like, for example:

pass in quick on em1 inet proto tcp from any to IP port = smtp

But with this approach you'd still be writing a parser, and there are a fair number of cases for it to cover.

And then there's brute force in the finest Unix traditions:

pftctl -s rules | tr ' ' '\012' | grep '^[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*$' | sort -u

We list all the rules, turn spaces into newlines to give us one word per line, and discard all words that don't look like IP addresses (and then discard duplicates). That pfctl separates words for us makes this both possible and the easiest, simplest approach. It's probably also the most reliable, since it's counting on almost nothing in pfctl's output.

So: sometimes brute force really is the best answer.

Sidebar: sorting IP addresses into ascending order

Because I keep having to do this: assuming that you have regularly formed IP addresses (using all decimal digits), the incantation on a modern version of sort to get them into proper ascending order by IP address is:

sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4

(As I keep forgetting, you have to explicitly tell sort that each sort key is only one field long. Otherwise it starts at field N and goes to the end of the line, which is not what we want.)