One of the drawbacks of operating a general purpose Unix environment for decades is that backwards compatibility can wind up causing you to get trapped in troublesome situations. In particular, the weight of backwards compatibility has wound up requiring us to configure our IMAP server environment in a way that causes various problems.

Unix IMAP servers generally have a setting for where the various IMAP mailboxes and folders are stored on disk. Back when we first set up UW-IMAP at least two decades ago, we wound up with a situation where UW-IMAP looked for and expected to find people's mail under their home directory, $HOME. People could manually put them in a subdirectory of $HOME if they wanted, or they could just drop them straight in $HOME.

(Based on old historical UW-IMAP source, this wasn't even a configuration option at the time. It was just what UW-IMAP was hard-coded to assume about mailbox and folder layout for everything except your INBOX mailbox.)

Some people left things as they were and had various mailboxes in $HOME. Some people decided to be neater and put everything in a subdirectory, but which subdirectory they picked was varied; some people used $HOME/Mail, some people used $HOME/IMAP, and so on. As we upgraded our IMAP server software over the years, eventually moving from UW-IMAP to Dovecot, we had to keep this configuration setting intact. If we dared change it, for example to say that all IMAP mailboxes and folders would henceforth be in $HOME/IMAP, we would be forcing lots of people to either change their client's IMAP configuration or relocate files and directories at the Unix level (and probably both for some people). This would have been a massive flag day and a massive disruption to our entire user base, not all of which are even on campus, with serious effects on their access to much of their email if things didn't go exactly right.

Now, there are two problems with an IMAP server that thinks your mailboxes and folders start in $HOME. The lesser problem is that if you ask the IMAP server for a list of all of your top level folders and mailboxes, you get a ls of $HOME (complete with all of your dotfiles). This is at least a bit annoying and it turns out that some software doesn't cope well with this, including our webmail system.

(We wound up having to force our webmail system to confine itself to a subfolder of the IMAP namespace and thus a subdirectory of $HOME. People who wanted to use webmail had to do some Unix and IMAP rearrangement, but at least this was an opt-in change; people who didn't care about webmail were unaffected.)

The more serious problem is that there is an IMAP operation that requires recursively finding all of your folders, subfolders, and mailboxes. This obviously requires recursing through the actual directory structure, and Dovecot will do this without limit and it follows directory symlinks. If you have a symlink somewhere under your $HOME that creates a cycle, Dovecot will follow this endlessly. If you have a symlink that escapes from your $HOME into the wider filesystem, Dovecot will also follow this and start trying to walk around (where it may hit someone else's symlink cycle). In either case, your Dovecot process basically hangs there and hammers away at our fileservers.

We're very fortunate in that very few clients seem to invoke this IMAP operation and so hung Dovecot processes using up CPU and NFS bandwidth are pretty uncommon. But they're not unknown; we get a few every so often. And it's mostly because of this backwards compatibility need.