Sysadmin use of email is often necessarily more or less interrupt driven

July 5, 2015

One of the things that people commonly do with virtual screens is to put their email client off in a separate virtual screen so they can ignore it and avoid having it interrupt them. As I mentioned when I wrote up my virtual screen usage, I don't try to cordon off email this way. Fundamentally this is because as a sysadmin, I feel my use of email is necessarily interrupt driven.

Allowing email to interrupt me certainly can derail my chain of thought when I'm coding or working on a hard problem. But at the same time it's absolutely necessary, because that email may carry news of an emergency or a high priority issue that I need to handle more or less right away. I almost never have the option of ignoring even the possibility of such things, so almost all of the time I have to allow email to interrupt me. The best I can do is contrive low distraction email monitoring so that when I'm in a flow state it distracts me as little as possible.

So I can stop there, right? No, not so fast. What this really argues is that email is a bad way of receiving high priority information like alerts. Because it mixes high priority information with much less important messages, I have to allow even unimportant things to interrupt me at least a bit just so I can figure out whether or not I can ignore them. If alerts and priority items came in through another channel, I could readily ignore email during high focus times.

(There are always going to be days where all I do is fiddle around with stuff and swat things as they come up; on those days, I'd read and handle everything right away.)

Of course the problem is that there is no good other channel today, at least for us. Oh, with work you can build such a thing (possibly outsourcing parts of it to companies who specialize in it), but there's very little in the way of a canned out of the box solution. Plus there's the problem of getting people use your new 'urgent things' channel when they have an urgent thing and of course not using it when they don't have an urgent thing (with the associated issue of having people know whether or not their issue is urgent).

(Life is likely somewhat easier if you can assume that everyone has a smartphone, perhaps by issuing them one, but that is not something that's true in our environment.)

Comments on this page:

By Ewen McNeill at 2015-07-05 06:49:05:

This "can't ignore email" is one of the reasons I have two email clients open all the time, on (basically) two email accounts: one for "interrupt" things, and one for "read when time permits" things. The "interrupt" one gets client emails, and some monitoring stuff, and has a noticeable "new email" alert on anything likely to be important (eg, anything not routine daily mail). The other one has a very subtle new email alert, and only on one specific mailbox. The trick is getting the right mail in the right mailbox... Which I still haven't perfected. But, eg all my list mail and much of my "I reported a bug and someone responded to it" mail, etc, goes into the less urgent one.

Also FWIW, I do put my email clients on separate virtual desktops. One each, in fact. Because I need to be able to get to it quickly, without hunting through a mass of windows clutter (I do tile windows where possible, but at some point it's not possible without making individual windows too small to be useful). Some "mostly email" tasks (eg, "review and comment on this document draft") get done in the email virtual desktop. But most tasks get done on some other virtual desktop, often one of the "current project" ones.

Finally as an observation, some organisations have an "if your problem is urgent, call this number" policy as their escalation procedure. Making everything else "less urgent". So someone can bump the priority by filing a ticket/email, and then calling to say "please look at ticket N now". Of course at that point you're back to interrupt driven phone calls... :-(


By cks at 2015-07-05 16:15:28:

My view is that most sysadmins are going to have to be interrupt driven by something. The big question is by what, and if you can get it down to something that interrupts you as little as possible (and thus only with things that really need it). Phone calls might be better than email if they only happen when the issue is urgent and don't happen otherwise, but that depends a lot on how people feel about them.

(For me a phone call is more disruptive than an email, even an urgent email, because phone calls are complete, immediate interruptions with no way of skimming them or whatever.)

By Perry Lorier at 2015-07-06 18:55:05:

My team has several types of automated interruptions:

User impact, or imminent user impact. You will need to take immediate action. For my team this means having hands on keyboards within 3 minutes. (We're unusual for such an extremely low limit.)
P0 Ticket
User impact is expected soon. You will need to take action within 30 minutes. If you don't claim the ticket within 30 minutes, then this turns into a Page.
Daily Ticket
Something is broken, and the automation is unable to recover the situation, but no user impact is expected unless there are more failures. You should investigate this within 1—2 business days.
Weekly Ticket
Something isn't right, you should investigate what's broken and repair it. Again, this is probably due to automation that has put in a good attempt, but ultimately failed to resolve the situation.

Most customer escalations are daily tickets, so should be resolved within 1—2 business days.

This means my daily routine is come in, burn through all the email that has arrived from the night before (People in Mountain View inconveniently write a lot of email...) and deal with any tickets that are assigned to me. If I'm rostered oncall, then I check every 15—30 minutes for tickets (while I remember to stretch, etc), and deal with any pages that arrive. I can do email in batches during the day as it suits me.

If an alert of any type is unactionable, then the action is fix the alert so that I won't get disturbed by it again. If I get annoyed at multiple alerts for the same thing, then I write/fix the automation so that this thing won't alert me again.

I shouldn't get more than 1—2 alerts per shift, otherwise it means that the technical debt is rising up again, and we need to go pay it back before we burn ourselves out.

Written on 05 July 2015.
« Googlebot and Feedfetcher are still aggressively grabbing syndication feeds
My Django form field validation and cleanup pain »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Jul 5 02:51:58 2015
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.