One reason you have a mysterious Unix file called
Suppose, one day, that you look at the
ls of some directory and
you notice that you have an odd file called '
2' (just the digit).
If you look at the contents of this file, it probably has nothing
that's particularly odd looking; in fact, it likely looks like
plausible output from a command you might have run.
Congratulations, you've almost certainly fallen victim to a simple typo, one that's easy to make in interactive shell usage and in Bourne shell scripts. Here it is:
echo hi >&2 echo oop >2
The equivalent typo to create a file called
1 is very similar:
might-err 2>&1 | less might-oop 2>1 | less
1 files created this way are often empty, although not always,
since many commands rarely produce anything on standard error.)
In each case, accidentally omitting the '
&' in the redirection
converts it from redirecting one file descriptor to another (for
echo to report something to standard error)
into a plain redirect-to-file redirection where the name of the
file is your target file descriptor number.
Some of the time you'll notice the problem right away because you don't get output that you expect, but in other cases you may not notice for some time (or ever notice, if this was an interactive command and you just moved on after looking at the output as it was). Probably the easiest version of this typo to miss is in error messages in shell scripts:
if [ ! -f "$SOMETHING" ]; then echo "$0: missing file $SOMETHING" 1>2 echo "$0: aborting" 1>&2 exit 1 fi
You may never run the script in a way that triggers this error condition, and even if you do you may not realize (or remember) that you're supposed to get two error messages, not just the 'aborting' one.
(After we stumbled over such a file recently, I grep'd all of
my scripts for '
>2' and '
>1'. I was relieved not to find
(For more fun with redirection in the Bourne shell, see also how to pipe just standard error.)
One downside of the traditional style of writing Unix manpages
A while back I wrote about waiting for a specific wall-clock time
in Unix, which according
you can do by using
clock_nanosleep with the
clock and the
TIMER_ABSTIME flag. This is fully supported on
and not supported on FreeBSD. But here's a
question: is it supported on Illumos-derived systems?
So, let us consult the Illumos
clock_nanosleep manpage. This manpage is very
much written in the traditional (corporate) style of Unix manpages,
high on specification and low on extra frills. This style either
invites or actively requires a very close reading,
paying very careful attention to both what is said and what is not
said. The Illumos manpage does not explicitly say that your sleep
immediately ends if the system's wall clock time is adjusted forward
far enough; instead it says, well:
If the flag
TIMER_ABSTIMEis set in the flags argument, the
clock_nanosleep()function causes the current thread to be suspended from execution until either the time value of the clock specified by
clock_idreaches the absolute time specified by the rqtp argument, [or a signal happens]. [...]
The suspension time caused by this function can be longer than requested because the argument value is rounded up to an integer multiple of the sleep resolution, or because of the scheduling of other activity by the system. [...] The suspension for the absolute
clock_nanosleep()function (that is, with the
TIMER_ABSTIMEflag set) will be in effect at least until the value of the corresponding clock reaches the absolute time specified by
rqtp, [except for signals].
On the surface, this certainly describes a fully-featured implementation
clock_nanosleep that behaves the way we want. Unfortunately,
if you're a neurotic reader of Unix manpages, all is not so clear.
The potential weasel words are 'the suspension ... will be in effect
at least until ...'. If you don't shorten
when the system clock jumps forward, you are technically having them
wait 'at least until' the clock reaches their timeout value, because
you sort of gave yourself room to have them wait (significantly) longer.
At the same time this is a somewhat perverse reading of the manpage,
partly because the first sentence of that paragraph alleges that the
system will only delay waking you up because of scheduling, which would
disallow this particular perversity.
To add to my uncertainty, let's look at the Illumos
manpage, which contains the
following eyebrow-raising wording:
If the flag
TIMER_ABSTIMEis set in the argument
timer_settime()behaves as if the time until next expiration is set to be equal to the difference between the absolute time specified by the
valueand the current value of the clock associated with
timerid. That is, the timer expires when the clock reaches the value specified by the
These two sentences do not appear to be equivalent for the case of
CLOCK_REALTIME clocks. The first describes an algorithm that
freezes the time to (next) expiration when
called, which is not proper
CLOCK_REALTIME behavior, and then
the second broadly describes correct
where the timer expires if the real time clock advances past it for
With all that said, Illumos probably fully implements
with proper handling of the system time being adjusted while you're
suspended or have a timer set. But its manpages never comes out and
say that explicitly, because that's simply not the traditional style
of Unix manpages, and the way they're written leaves me with
uncertainty. If I cared about this, I would have to write a test
program and then run it on a machine where I could set the system
time both forward and backward.
This fault is not really with these specific Illumos manpages, although some elements of their wording aren't helping things. This is ultimately a downside to the terse, specification-like traditional style of Unix manpages. Where every word may count and the difference between 'digit' and 'digits' matters, you sooner or later get results like this, situations where you just can't tell.
(Yes, this would be a perverse implementation and a weird way of writing
the manpages, but (you might say) perhaps the original Solaris corporate
authors really didn't want to admit in plain text that Solaris didn't
have a complete implementation of
Also, I'm sure that different people will read these manpages
differently. My reading is unquestionably biased by knowing that
clock_nanosleep support is not portable across all Unixes, so I
started out wondering if Illumos does support it. If you start reading
these manpages with the assumption that of course Illumos supports it,
then you get plenty of evidence for that position and all of the wording
that I'm jumpy about is obviously me being overly twitchy.
popen() API works but more complex versions blow up
Years ago I wrote about a long-standing Unix issue with more
sophisticated versions of
popen(); my specific
example was writing a large amount of stuff to a subprogram through
a pipe and then reading its output, where both sides stall trying
to write to full pipes. Of course this is not the only way to have
this problem bite you, so recently I ran across Andrew Jorgensen's
A Tale of Two Pipes (via), where
the same problem comes up when a subprogram writes to both standard
output and standard error and you consume them one at a time.
Things like Python's
subprocess module and many other
imitators generally trace their core idea back to the venerable Unix
popen(3) library function, which first appeared in V7 Unix.
popen() itself does not actually have this problem; only
more sophisticated and capable interfaces based on it do.
popen() doesn't have the problem is straightforward
and points to the core problem with more elaborated versions of the
popen() doesn't have a problem because it only gives you
a single IO stream, either the sub-program's standard input or its
standard output. More sophisticated APIs give you multiple streams,
and multiple streams are where you get into trouble. You get into
trouble because more sophisticated APIs with multiple streams are
implicitly pretending that the streams can be dealt with independently
and serially, ie that you can fully process one stream before looking
at another one at all. As A Tale of Two Pipes makes clear, this
is not so. In actuality the streams are inter-dependent and have
to be processed together, although Unix pipe buffers can hide this
from you for a while.
Of course you can handle the streams properly yourself, resorting
poll() or some similar measure. But you shouldn't have to
remember to do that, partly because as long as you have to take
additional complex steps to make things work right, people are going
to be forgetting this requirement. In the name of looking simple
and generic, these APIs have armed a gun that is pointed straight
at your feet. A more honest API would make the inter-dependency
clear, perhaps by returning a
Subprocess object that you registered
callbacks on. Callbacks have a bad reputation but they at least
make it clear that things can (and will) happen concurrently, instead
of one stream being fully handled before another stream is even
(Go has an interesting approach to the problem that is sort of half
solution and half not. In its core
os/exec API for this, you you provide streams which
will be read from or written to asynchronously. However there are
that give you a more traditional 'here is a stream' interface and
with it the traditional problems.)
Sidebar: Why people keep creating these flawed subprogram APIs on Unix
These APIs keep getting created because they're attractive. How the
API appears to behave (ie, without the deadlock issues) is how
people often want to deal with subprograms. Most of the time you're
not interacting with them step by step, sending in some input and
collecting some output; instead you're sending in the input,
collecting the output, and maybe collecting standard error as well
in case something blew up. People don't want to write
loops or callbacks or anything complicated, because concurrency is at
least annoying. They just want the simple API to work.
Possibly libraries should make the straightforward user code work by handling all of the polling and so on internally and being willing to buffer unlimited amounts of standard output and standard error. This would probably blow up less often than the current scheme does, and you could provide various options for how much to buffer and how to deal with overflow for advanced users.
Digging into BSD's choice of Unix group for new directories and files
I have to eat some humble pie here. In comments on my entry on
chmod failure, Greg A.
Woods pointed out that FreeBSD's behavior of creating everything
inside a directory with the group of the directory is actually
traditional BSD behavior (it dates all the way back to the 1980s),
not some odd new invention by FreeBSD. As traditional behavior it
makes sense that it's explicitly allowed by the standards, but
I've also come to think that it makes sense in context and in
general. To see this, we need some background about the problem
In the beginning, two things were true in Unix: there was no
mkdir() system call, and processes
could only be in one group at a time. With processes being in only
one group, the choice of the group for a newly created filesystem
object was easy; it was your current group. This was felt to be
sufficiently obvious behavior that the V7
doesn't even mention it.
(The actual behavior is implemented in the kernel in
Now things get interesting. 4.1c BSD seems to be where
is introduced and where
creat() stops being a system call
and becomes an option to
It's also where processes can be in multiple groups for the first
time. The 4.1c BSD
open(2) manpage is silent about the group of
newly created files, while the
mkdir(2) manpage specifically
claims that new directories will have your effective group (ie, the
V7 behavior). This is actually wrong. In both
maknode() in ufs_syscalls.c,
the group of the newly created object is set to the group of the
parent directory. Then finally in the 4.2 BSD
the group of the new directory is correctly documented (the 4.2
continues to say nothing about this). So BSD's traditional behavior
was introduced at the same time as processes being in multiple groups,
and we can guess that it was introduced as part of that change.
When your process can only be in a single group, as in V7, it makes perfect sense to create new filesystem objects with that as their group. It's basically the same case as making new filesystem objects be owned by you; just as they get your UID, they also get your GID. When your process can be in multiple groups, things get less clear. A filesystem object can only be in one group, so which of your several groups should a new filesystem object be owned by, and how can you most conveniently change that choice?
One option is to have some notion of a 'primary group' and then
provide ways to shuffle around which of your groups is the primary
group. One problem with this is that it's awkward and error-prone
to work in different areas of the filesystem where you want your
new files and directories to be in different groups; every time you
cd around, you may have to remember to change your primary group.
If you move into a collaborative directory, better shift (in your
shell) to that group;
cd back to
$HOME, or simply want to write
a new file in
$HOME, and you'd better remember to change back.
Another option is the BSD choice of inheriting the group from
context. By far the most common case is that you want your new files
and directories to be created in the 'context', ie the group, of
the surrounding directory. If you're working in
$HOME, this is
your primary login group; if you're working in a collaborative area,
this is the group being used for collaboration. Arguably it's a
feature that you don't even have to be in that group (if directory
permissions allow you to make new files). Since you can
directories that you own, this option also gives you a relatively
easy and persistent way to change which group is chosen for any
If you fully embrace the idea of Unix processes being in multiple groups, not just having one primary group and then some number of secondary groups, then the BSD choice makes a lot of sense. And for all of its faults, BSD tended to relatively fully embrace its changes (not totally, perhaps partly because it had backwards compatibility issues to consider). While it leads to some odd issues, such as the one I ran into, pretty much any choice here is going to have some oddities. It's also probably the more usable choice in general if you expect much collaboration between different people (well, different Unix logins), partly because it mostly doesn't require people to remember to do things.
(I know that on our systems, a lot of directories intended for collaborative work tend to end up being setgid specifically to get this behavior.)
chmod can fail for interesting reasons
I'll start by presenting this rather interesting and puzzling failure in illustrated form:
; mkdir /tmp/newdir ; chmod g+s /tmp/newdir chmod: /tmp/newdir: Operation not permitted
How can I not be able to make this chmod change when I just made the directory and I own it? For extra fun, some people on this particular system won't experience this problem, and in fact many of them are the people you might report this problem to, namely the sysadmins.
At first I wondered if this particular
/tmp filesystem disallowed
setuid and setgid entirely, but it turned out to be not that
; ls -ld /tmp/newdir drwxr-xr-x 2 cks wheel 512 May 3 00:35 /tmp/newdir
This at least explains why my chmod attempt failed. I'm not in group
wheel, and for good reasons you can't make a file setgid to a
group that you're not a member of. But how on earth did my newly created
/tmp wind up in group wheel, a group I'm not a member
of? Well, perhaps someone made
/tmp setgid, so all directories created
in it inherited its group (presumably group
wheel). Let's see:
; ld -ld /tmp drwxrwxrwt 157 root wheel 11776 May 3 00:41 /tmp
/tmp is indeed group
wheel, it has perfectly ordinary
permissions (mode 777 and sticky ('
t'), so you can only delete or
rename your own files). There's no setgid to be seen.
The answer to this mystery is that this is a FreeBSD machine, and
on FreeBSD, well, let's quote the
The directory's owner ID is set to the process's effective user ID. The directory's group ID is set to that of the parent directory in which it is created.
And also the section of the
open(2) manpage that deals
with creation of new files:
When a new file is created it is given the group of the directory which contains it.
In other words, on FreeBSD all directories have an implicit setgid
bit. Everything created inside them (whether directories or files)
inherits the directory's group. Normally this is not a problem and
you'll probably never notice, but
/var/tmp) are special
because they allow everyone to create files and directories in them,
and so there are a lot of people making things there who are not a
member of the directory's group.
(The sysadmins usually are members of group
wheel, though, so things
will work for them. This should add extra fun if a user reports the
general chmod issue as a problem, since sysadmins can't reproduce it as
You might think that this is an obscure issue that no one will ever care about, but actually it caused a Go build failure on FreeBSD for a while. Tracking down the problem took me a while and a bunch of head scratching.
PS: arguably GID 0 should not be group
wheel but instead something
else that only
root is a member of and
wheel should be a
completely separate group. To have group
wheel used for group
ownership as well as
su access to
root is at least confusing.
Some versions of sort can easily sort IPv4 addresses into natural order
Every so often I need to deal with a bunch of IPv4 addresses, and
it's most convenient (and best) to have them sorted into what I'll
call their natural ascending order. Unfortunately for sysadmins,
the natural order of IPv4 addresses is not their lexical order (ie
sort will give you), unless you zero-pad all of their octets.
In theory you can zero pad IPv4 addresses if you want, turning
18.104.22.168 into 058.172.099.001, but this form has two flaws; it
looks ugly and it doesn't work with a lot of tools.
(Some tools will remove the zero padding, some will interpret
zero-padded octets as being in octal instead of decimal, and some
will leave the leading zeros on and not work at all;
dig -x is
one interesting example of the latter. In practice, there are
much better ways to deal with this problem and people who zero-pad
IPv4 addresses need to be politely corrected.)
Fortunately it turns out that you can get many modern versions of
sort to sort plain IPv4 addresses in the right order. The trick is
to use its
-V argument, which is also known as
in at least GNU coreutils. Interpreting IPv4 addresses as version
numbers is basically exactly what we want, because an all-numeric
MAJOR.MINOR.PATCH.SUBPATCH version number sorts in exactly the same
way that we want an IPv4 A.B.C.D address to sort.
Unfortunately as far as I know there is no way to sort IPv6 addresses into a natural order using common shell tools. The format of IPv6 addresses is so odd and unusual that I expect we're always going to need a custom program for it, although perhaps someday GNU Sort will grow the necessary superintelligence.
This is a specific example of the kind of general thinking that you need in order to best apply Unix shell tools to your problems. It's quite helpful to always be on the lookout for ways that existing features can be reinterpreted (or creatively perverted) in order to work on your problems. Here we've realized that sort's idea of 'version numbers' includes IPv4 addresses, because from the right angle both they and (some) version numbers are just dot-separated sequences of numbers.
PS: with brute force, you can use any version of
sort that supports
-k to sort IPv4 addresses; you just need the right magic
arguments. I'll leaving working them out (or doing an Internet
search for them) as an exercise for the reader.
PPS: for the gory details of how GNU sort treats version sorting,
see the Gnu sort manual's
section on details about version sort.
Okay, technically it's
ls's section on version sorting. Did you know
that GNU coreutils ls can sort filenames partially based on version numbers?
I didn't until now.
(This is a more verbose version of this tweet of mine, because why should I leave useful stuff just on Twitter.)
Sidebar: Which versions of
sort support this
When I started writing this entry, I assumed that
sort -V was a
GNU coreutils extension and would only be supported by the GNU
coreutils version. Unixes with other versions (or with versions
that are too old) would be out of luck. This doesn't actually appear
to be the case, to my surprise.
Based on the GNU Coreutils NEWS file, it appears that '
appeared in GNU coreutils 7.0 or 7.1 (in late 2008 to early 2009).
The GNU coreutils sort is used by most Linux distributions, including
all of the main ones, and almost anything that's modern enough to be
getting security updates should have a version of GNU sort that is
recent enough to include this.
Older versions of FreeBSD appear to use an old version of GNU
coreutils sort; I have access to a FreeBSD 9.3 machine that reports
/usr/bin/sort is GNU coreutils sort 5.3.0 (from 2004,
apparently). Current versions of FreeBSD and OpenBSD have switched
to their own version of
sort, known as version '2.3-FreeBSD', but
this version of
sort also supports
-V (I think the switch
happened in FreeBSD 10, because a FreeBSD 10.3 machine I have access
to reports this version). Exactly how
-V orders things is probably
somewhat different between GNU coreutils sort and FreeBSD/OpenBSD
sort, but it doesn't matter for IPv4 addresses.
/usr/bin/sort is very old, but I know that OmniOS
/usr/gnu/bin/sort as standard and really you want
early in your
$PATH anyways. Life is too short to deal with ancient
Solaris tool versions with ancient limitations.
Wayland is now the future of Unix graphics and GUIs
The big Unix graphics news of the past week is that Ubuntu threw in the towel on their Unity GUI and with it their Mir display server (see the Ars story for more analysis). I say 'Unix' instead of 'Linux' here because I think this is going to have consequences well beyond Linux.
While there was a three-way fight for the future between Wayland, Ubuntu's Mir, and the default of X, it was reasonably likely that support for X was going to remain active in things like Firefox, KDE, and even Gnome. As a practical matter, Mir and Wayland were both going to support X programs, so if you targeted X (possibly as well as Wayland and/or Mir) you could run on everything and people would not be yelling at you and so on. But, well, there isn't a three-way fight any more. There is only X and Wayland now, and that makes Wayland the path forward by default. With only one path forward, the pressure for applications and GUI environments to remain backwards compatible to X is going to be (much) lower. And we already know how the Gnome people feel about major breaking changes; as Gnome 3 taught us, the Gnome developers are perfectly fine with them if they think the gain is reasonable.
In short: running exclusively on Wayland is the future of Gnome and Gnome-based programs, which includes Firefox; I suspect that it's also the future of KDE. It's not an immediate future, but in five years I suspect that it will be at least looming if not arriving. At that point, anyone who is not running Wayland will not be getting modern desktop software and programs and sooner or later won't be getting browser security fixes for what they currently have.
People run desktop software on more Unixes than just Linux. With Gnome and important desktop apps moving to Wayland, those Unixes face a real problem; they can live with old apps, or they can move to Wayland too. FreeBSD is apparently working seriously on Wayland support (cf), and at one point a Dragonfly BSD developer had Wayland running there. OpenBSD? Don't hold your breath. Solaris? That's up to Oracle these days but I don't really expect it; it would be a lot of work and I can't imagine that Oracle has many customers who will pay for it. Illumos? Probably not unless someone gets very energetic.
With that said, old X programs and environments are not going to
suddenly go away. Fvwm will be there for years
or decades to come, for example, as will
xterm and any number of
other current X programs and window managers. But people who are
stuck in X will also be increasingly stuck in the past, unable to
run current versions of more and more programs.
(For some people, this will be just fine. We're probably going to see a fairly strong sorting function among the free Unixes for what sort of person winds up where, which is going to make cultural issues even more fun than usual.)
PS: Some people may sneer at 'desktop software and programs', but this category includes quite a lot of things that are attractive but by and large desktop agnostic, like photography programs, Twitter clients, and syndication feed readers. Most modern graphical programs on Unix are built on top of some mid-level toolkit like GTK+ or QT, not on basic X stuff, because those mid-level toolkits make it so much faster and easier to put together GUIs. If and when those toolkits become Wayland-only and the latest versions of the programs move to depend on recent versions of the toolkits, the programs become Wayland-only too.
Why the modern chown command uses a colon to separate the user and group
In the beginning, all
chown(1) did was change the owner of a file;
if you wanted to change a file's group too, you had to use
as well. This is actually more unusual than I realized before I started
to write this entry, because even in V7 Unix the
chown(2) system call
itself could change both user and group, per the V7
chown(1) to only changing the owner did make the command
pretty simple, though.
By the time of 4.1c BSD,
chown(1) had become
because, per the manual page:
Only the super-user can change owner,
in order to simplify as yet unimplemented accounting procedures.
(The System V line of Unixes would retain an unrestricted
system call for some time and thus I believe they kept the
command in section 1, for general commands anyone could use.)
In 4.3 BSD, someone decided that
chown(8) might as well let you change
the group at the same time, to match the system call. As the manual page
covers, they used this syntax:
/etc/chown [ -f -R ] owner[.group] file ...
That is, to chown a file to user
staff, you did
chown cks.staff file'.
This augmented version of the
chown command was picked up by various
Unixes that descended from 4.x BSD, although not immediately (like many
things from 4.3 BSD, it took a while to propagate around). Sometimes
this was the primary version of
chown, found in
/usr/bin or the
like; sometimes this was a compatibility version, in
through fairly late, for example). Depending on how you set up your
$PATH on such systems, you could wind up using this version of
and thus get used to having 'user:group' rejected as an error.
Then, when it came time for POSIX to standardize this, someone woke
up and created the modern syntax for changing both owner and group
at once. As seen in the Single Unix Specification for
this is '
chown owner[:group] file', ie the separator is now a
colon. Since POSIX and the SUS normally standardized existing
practice (where it actually existed), you might wonder why they
changed it. The answer is simple: a colon is not a valid character
in a login, while a dot is.
Sure, dots are unusual in Unix logins in most places, but they're
legal and they do show up in some environments (and they're legal
in group names as well). Colons are outright illegal unless you
like explosions, fundamentally because they're the field separator
/etc/group. The SUS manpage
actually has an explicit discussion of this in the RATIONALE section,
although it doesn't tell you what it means by 'historical
(The SUS manpage also discusses a scenario where using
chgrp separately isn't sufficient, and you have to make the change
in a single
chown() system call.)
PS: Since I think I ran into this dot-versus-colon issue on our
old Solaris 10 fileservers, I
/usr/bin in my
$PATH there. I
generally prefer UCB versions of things to the stock Solaris versions,
but in this case it tripped me up.
PPS: It turns out the GNU chown accepts the dot form as well provided that it's unambiguous, although this is covered only in the chown info file, and is not mentioned in the normal manpage.
What should it mean for a system call to time out?
I was just reading Evan Klitzke's Unix System Call Timeouts (via) and among a number of thoughts about it, one of the things that struck me is a simple question. Namely, what should it mean for a Unix system call to time out?
This question may sound pointlessly philosophical, but it's actually
very important because what we expect a system call timeout to mean
will make a significant difference in how easy it would be to add
system calls with timeouts. So let's sketch out two extreme versions.
The first extreme version is that if a timeout occurs, the operation
done by the system call is entirely abandoned and undone. For
example, if you call
rename("a", "b") and the operation times
out, the kernel guarantees that the file
a has not been renamed
b. This is obviously going to be pretty hard, since the kernel
may have to reverse partially complete operations. It's also not
always possible, because some operations are genuinely irreversible.
write() data to a pipe and time out partway through doing
so (with some but not all data written), you cannot reach into the
pipe and 'unwrite' all of the already sent data; after all, some
of it may already have been read by a process on the other side of
The second extreme version is that having a system call time out merely causes your process to stop waiting for it to complete, with no effects on the kernel side of things. Effectively, the system call is shunted to a separate thread of control and continues to run; it may complete some time, or it may error out, but you never have to wait for it to do either. If the system call would normally return a new file descriptor or the like, the new file descriptor will be closed immediately when the system call completes. In practice implementing a strict version of this would also be relatively hard; you'd need an entire infrastructure for transferring system calls to another kernel context (or more likely, transplanting your user-level process to another kernel context, although that has its own issues). This is also at odds with the existing system calls that take timeouts, which generally result in the operation being abandoned part way through with no guarantees either way about its completion.
(For example, if you make a non-blocking
connect() call and then
select() to wait for it with a timeout, the kernel does not
guarantee that if the timeout fires the
connect() will not be
completed. You are in fact in a race between your likely
of the socket and the connection attempt actually completing.)
The easiest thing to implement would probably be a middle version.
If a timeout happens, control returns to your user level with a
timeout indication, but the operation may be partially complete and
it may be either abandoned in the middle of things or completed for
you behind your back. This satisfies a desire to be able to bound
the time you wait for system calls to complete, but it does leave
you with a messy situation where you don't know either what has
happened or what will happen when a timeout occurs. If your
times out, the directory may or may not exist when you look for it,
and it may or may not come into existence later on.
(Implementing timeouts in the kernel is difficult for the same reason that asynchronous IO is hard; there is a lot of kernel code that is much simpler if it's written in straight line form, where it doesn't have to worry about abandoning things part way through at essentially any point where it may have to wait for the outside world.)
Modern X Windows can be a very complicated environment
I mentioned Corebird, the GTK+ Twitter client the other day, and generally positive. That was on a logical weekend. The next day I went in to the office, set up Corebird there, and promptly ran into a problem: I couldn't click on links in Tweets, or rather I could but it didn't activate the link (it would often do other things). Corebird wasn't ignoring left mouse clicks in general, it's just that they wouldn't activate links. I had not had this problem at home (or my views would not have been so positive. I use basically the same fvwm-based window manager environment at home and at work, but since Corebird is a GTK+ application and GTK+ applications can be influenced by all sorts of magic settings and (Gnome) setting daemons, I assumed that it was something subtle that was different in my work GTK+/Gnome environment and filed a Fedora bug in vague hopes. To my surprise, it turned out to be not merely specific to fvwm, but specific to one aspect of my particular fvwm mouse configuration.
The full version is in the thread in the fvwm mailing list, but normally when you click and release a button, the X server generates two events, a ButtonPress and then a ButtonRelease. However, if fvwm was configured in a way such that it might need to do something with a left button press, a different set of events was generated:
- a LeaveNotify with mode NotifyGrab, to tell Corebird that the mouse pointer had been grabbed away from it (by fvwm).
- an EnterNotify with mode NotifyUngrab, to tell Corebird 'here is your mouse pointer back because the grab has been released' (because fvwm was passing the button press through to Corebird).
- the ButtonPress for the mouse button.
The root issue appears to be that something in the depths of GTK+ takes the LeaveNotify to mean that the link has lost focus. Since GTK+ doesn't think the link is focused, when it receives the mouse click it doesn't activate the link, but it does take other action, since it apparently still understands that the mouse is being clinked in the text of the GtkLabel involved.
If you think that this all sounds very complex, yes, exactly. It is. X has a complicated event model to start with, and then interactions with the window manager add extra peculiarities on top. The GTK+ libraries are probably strictly speaking in the wrong here, but I also rather suspect that this is a corner case that the GTK+ programmers never imagined, much less encountered. In a complex environment, some possibilities will drop through the cracks.
(If you want to read a high level overview of passive and active (mouse button) grabs, see eg this 2010 writeup by Peter Hutter. Having read it, I feel like I understand a bit more about what fvwm is doing here.)
By the way, some of this complexity is an artifact of the state of computing when X was created, specifically that both computers and networking were slow. Life would be simpler for everyone if all X events were routed through the window manager and then the window manager passed them on to client programs as appropriate. However, this would require all events to pass through an extra process (and possibly an extra one or two network hops), and in the days when X was young this could have had a real impact on overall responsiveness. So X goes to a great deal of effort to deliver events directly to programs whenever possible while still allowing the window manager to step in.
(My understanding is that in Wayland, the compositor handles all events and passes them to clients as it decides. The Wayland compositor is a lot more than just the equivalent of an X window manager, but it fills that role, and so in Wayland this issue wouldn't come up.)