Wandering Thoughts archives

Our revised Dovecot IMAP configuration migration plans (and processes)

Back at the start of January, I wrote up the goals and problems of our Dovecot IMAP migration, and in an appendix at the end I outlined what became our initial migration plans. We would build an entirely new Dovecot server that was set up with people's IMAP mail folder storage being a subdirectory of their $HOME, say $HOME/mail (call this the IMAP root), and then we would get people to move to this server one by one. Migration would require them to change their clients and might require them (or us) to move files in Unix. Eventually we would tell the remaining holdouts that we were just going to turn off the old IMAP server and they had to migrate now.

Initially, the great virtue I saw in this plan was that it was entirely user driven and didn't require us to do anything. The users did everything, could go at their own speed, and were completely responsible for what happened. In an environment where we couldn't count on clients using IMAP subscriptions so we could know what people's mailboxes actually were, things had to be user-driven anyway, and we generally try to stay out of doing per-user things because it doesn't scale; we have a lot of users and not very many people looking after our central systems (including the IMAP server).

As we talked more and more about this, we realized that the central problem with this plan is that everyone had to migrate and this involved the users doing things (often at the Unix level), or getting someone to help them. As mentioned, we have a lot of users, and some of them are quite important (eg, professors) and can't just be abandoned to their fate. There was no way to make this not be disruptive to people. At the same time, most of our users were not causing any problems, which meant that we'd be forcing a lot of people to do disruptive things (on all of their devices, better not miss one) to deal with a problem created by a much smaller number of users.

If this was the only way to deal with things, we might still have gone ahead with it. But as I sort of alluded to in passing in the January entry, it's possible to do this on a per-user basis in Dovecot using a shell script (see the bottom of MailLocation). After we talked it over, we decided that this was the way we wanted to handle the migration to people's IMAP sessions being confined to a subdirectory of their $HOME; it would be done on a per-user basis and we'd directly target high-priority problem cases. The vast majority of our current users would forever stay un-migrated, while new users would be set up to be confined to a $HOME subdirectory from the start (ie, using the new IMAP root).

As much as possible, we wanted this migration to be transparent to users (or at least important ones). That meant that the IMAP mailbox names as seen by the clients couldn't change, and that meant that no matter what we were going to have to move files around; there's no other way for this to be transparent to clients when you change the IMAP root. Given that, it wasn't important to pick a new IMAP root that people already used for mailboxes, so we picked $HOME/IMAP for various reasons (including that calling it this made it clear what it was for).

Since this plan means that we're moving user mailboxes around at least some of the time (in order to migrate problem users), knowing what those mailboxes were became important enough to get us to hack some mailbox logging into Dovecot. Having this information has been extremely reassuring. Even when it just duplicates the information in a user's .subscriptions file, it also confirms that that information is accurate and complete.

We started out with plans for a two-stage operation for most users, where we'd first tell them to move all of their IMAP mailboxes under 'mail/' in their client (ie, $HOME/mail) before some deadline, then at the deadline we'd make $HOME/mail into $HOME/IMAP/mail and flip the server setting that made $HOME/IMAP their IMAP root. In practice it's turned out to be easier to do the file moving ourselves, based on both .subscriptions and the logs, so our current approach is to just tell various users 'unless you object, at time X we'll be improving your IMAP client experience by ...' and then at time X we do everything ourselves. It's been a little bit surprising how few actual active mailboxes some of these users have, especially relative to how much of an impact they've been having on the server.

(This genuinely does improve the IMAP client experience for people, for obvious reasons. An IMAP client that is scanning all of your $HOME and maybe opening all the files there is generally not a responsive client, not if your $HOME is at all big.)

PS: Although I haven't been writing about it here on Wandering Thoughts until recently, our IMAP situation has been consuming a lot of my attention and time at work. It's turned into a real learning experience in several ways.