Making the Linux kernel shut up about segfaulting user programs

July 14, 2010

Relatively modern versions of the Linux kernel on some architectures default to logging a kernel message about any process that receives various unhandled signals. The exact details depend on the architecture, but I think it is common for the architectures to log messages about unhandled SIGSEGVs, ie segmentation faults.

(Most architectures are capable of logging these messages if the feature is turned on, but not all architectures turn it on by default. It appears that x86 (both 32-bit and 64-bit) and SPARC default to having it on, and PowerPC and S390 default to having it off. SPARC support for this is recent, having only been added around March of this year)

I find these messages rather irritating, given that we run multiuser systems that are full of users running various programs, some of them locally written and some of them just buggy. They clutter up our kernel message logs, making it harder to notice other things, and there is no useful information for us in them.

(The kernel message is ratelimited so that it can't flood the logs, but given the relatively low volume of kernel messages in general it can easily be the dominating message.)

On all architectures that support this, whether it is on or off is controlled by the debug.exception-trace sysctl (aka /proc/sys/debug/exception-trace); a value of 1 means that the kernel logs messages, a 0 means that it doesn't. On the S390, there is a second sysctl that also controls it, kern.userprocess_debug, which is presumably still there for historical reasons.

(This is the kind of entry that I write so that I can find it later.)

Sidebar: the kernel code itself

The kernel sorce code variable that controls this behavior is show_unhandled_signals. It's almost entirely used (and defined) in architecture dependent code, which is why it has different defaults and behaviors on different architectures. There is one conditional bit in general kernel code, in kern/sysctl.c, to define the sysctl itself.

Comments on this page:

From at 2010-07-26 21:35:09:

I agree this is annoying. On a similar note, I am annoyed by the fact that programmers (including myself) are guilty of using words ranging from "can't", "don't", and "won't" to "error", "warn", and "problem" to indicate bad things. Since there is no standard way to detect an error in the logs, I wrote a log analysis program called petit ( to help with this problem.

In the case of this kernel message, it would reduce it to a single line, which could easily be ignored.

Scott M (fatherlinux)

Written on 14 July 2010.
« The challenges of shared spares in RAID arrays
Realistic blog usability »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Jul 14 16:36:30 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.