Wandering Thoughts archives

You want to turn console blanking off on your Linux servers

Let's start with the tweets:

@thatcks: Everyone should strongly consider adding 'consoleblank=0' to the kernel command line on your Linux servers. #sysadmin
@thatcks: The Linux kernel blanking the console screen is both unnecessary and dangerous on modern servers and modern setups. You want it off.

By default if you leave a Linux machine sitting idle at a text console, the kernel will blank the display after a while (I believe it's normally ten minutes of inactivity); Linux has probably done this since the very early days. Back in the time of CRT displays this made a reasonable amount of sense, because it avoided burning in the login prompt or whatever other static text was probably on the screen. Screen burnin is not really an issue in the modern age with LCDs, and it's even less of an issue with modern servers that spend a close approximation to all of their time without a display plugged in at all.

The problem with this console blanking is that it is a kernel function and thus the kernel has to reverse it. More specifically, the kernel has to be fairly alive and responding to the keyboard in order to unblank the screen. There are plenty of ways to get a kernel so hung that it is not alive enough to do this, at which point any helpful diagnostic messages the kernel may have printed on its way down are lost, locked away behind that blank screen. We have had this happen to us more than once.

And that is why you don't want your servers to ever blank their consoles; it's not getting you anything worthwhile and it can really hurt you. The best way to disable it is, as I tweeted, to add 'consoleblank=0' to the kernel command line arguments.

(Some people fiddle around with 'setterm -blank 0' in various ways but the kernel argument is more sure and easier.)

(I found out about 'consoleblank=0' and a bunch of additional useful information from this stackexchange question and its answers, when I finally decided to see if we could disable console blanking on our new iSCSI backends. I admit that my motivation for it was rather more petty than the reason here; a blank console can sometimes make their KVM-over-IP Java program freak out in a really irritating way and I was getting tired of that happening to my sessions.)

Some notes on bisecting a modified Firefox source base with Mercurial

Suppose, not hypothetically, that you maintain your own copy of the Firefox master source (aka 'Nightly') with private modifications on top of the Mozilla version. Of course you don't commit your modifications, because that would lead to a huge tangle of merges over time. Now suppose that Mozilla breaks something and you want to use Mercurial bisection to find it.

The first thing you need is to figure out the last good version. What I do is I don't run my modified Firefox version directly out of the build directory; instead I periodically make an install tarball and unpack it elsewhere (and then keep the last few ones when I update it, so I can revert in case of problems). Among other things this tarball copy has an application.ini file, which for my builds includes a SourceStamp= value that gives the Mercurial commit identifier that the source was built from.

So we start the procedure by setting the range bounds:

hg bisect --good 606848e8adfc
hg bisect --bad tip

Since I'm carrying local modifications this will generally report something like:

Testing changeset 192456:d2e7bd70dd95 (1663 changesets remaining, ~10 tests)
abort: uncommitted changes

So now I need to explicitly check out the named changeset. If I skip this step Mercurial won't complain (and it will keep doing future 'hg bisect' operations without any extra complaints), but what I'm actually doing all of the time is building the tip of my repo. This is, as they say, not too useful. So:

hg checkout d2e7bd70dd95

This may print messages about merging changes in my changed files, which is expected. In general Mercurial is smart enough to get merging my changes in right unless something goes terribly wrong. Afterwards I build and test and do either 'hg bisect --good' or 'hg bisect --bad' followed by another 'hg checkout <ver>'.

(If I remember I can use the '-U' argument to 'hg bisect' so it doesn't attempt the checkout and abort with an error, but enhh. I actually think that having the error is handy because it reminds me that I need extra magic and care.)

In some cases even the 'hg checkout' may fail with the uncommitted changes error message. In this case I need to drop my changes and perhaps re-establish them later. The simple way is:

hg shelve
hg checkout ...

Perhaps I should routinely shelve all of my changes at the start of the bisection process, unless I think some of them are important for the testing I'm doing. It would cut down the hassle (and shelving them at the start would make it completely easy to reapply them at the end, since they'd be taken from tip and reapplied to tip).

After the whole bisection process is done, I need to cancel it and return to the tip of the tree:

hg bisect --reset
hg checkout tip
# if required:
hg unshelve
# optional but customary:
hg pull -u

(This is the sort of notes that I write for myself because it prevents me from having to reverse engineer all of this the next time around.)

Sidebar: Some related Mercurial bits I want to remember

The command to tell me what checkout I am on is 'hg summary' aka 'hg sum'. 'hg status' doesn't report this information; it's just for file status. This correctly reports that the current checkout hasn't changed when a 'hg bisect ...' command aborts due to uncommitted changes.

I don't think there's an easy command to report whether or not a bisection is in progress. The best way to check is probably:

hg log -r 'bisect(current)'

If there's no output, there's no bisection in flight.

(I believe I've left bisections sitting around in the past by omitting the 'hg bisect --reset'. If I'm right, things like 'hg pull -u' and so on won't warn me that theoretically there is a bisection running.)