The problem with overly verbose package installation

March 25, 2010

We just applied a kernel update to all of our Ubuntu LTS machines. This is a quite verbose process; on the machine I captured logs for, it produces no less than 40 lines of status output (and another 14 lines before things start installing). And this count understates things because some of those are very long lines that wrap around on any real terminal.

(The kernel updating was done with 'apt-get install ...'.)

I'm sure that the authors of the Ubuntu kernel package and all of the management programs involved in this update process thought that they had very important things to tell people, and some of this is because we have a fair number of kernels installed as a result of running these systems for a while; we might get about 14 lines less if we went around removing old kernels by hand. Sadly, all of those people are wrong, and I will tell you why: I lied about the line count.

On a few of our machines, including the one I have logs for, applying the kernel updates didn't produce 40 lines of output; it produced 41. The extra line in the middle was an error report (possibly not a significant one; it's not clear and the machines all rebooted with the new kernel). How many machines? I don't know. The difference between 40 lines and 41 lines is not very easy to notice, especially in repetitive text that you see ten or twenty copies of.

And this is the problem: overly verbose package installation hides problems. Problem reports are a needle in a haystack, and the haystack is your regular output; the more regular output, the less the needle stands out. Produce almost no output and the needle is glaringly obvious.

The other factor is how simple and patterned your regular output is. Yum has a fair bit of output by default but it has a very simple pattern (it's progress bar after progress bar), so it is relatively easy to pick out exceptional messages (not as easy as it would be if there was less output, though). The Ubuntu kernel update process's regular output is unpatterned, normal text, so a problem message does not particularly stand out unless you look carefully.

(By 'patterned' here I mean the abstract visual patterns, divorced of particular words and so on. Humans are quite good at spotting breaks in regular visual patterns; we are much less good at spotting particular things in a random mash of visual stuff.)

It's my jaundiced opinion that the problem here is a cultural one. I doubt there's anything intrinsic about apt-get that requires things to be verbose; instead, Debian and now Ubuntu has created a culture where it is both acceptable and routine for package installation to produce a lot of status messages and reports about completely routine events. At this point, the keepers of the culture might even view it as somehow wrong for packages to be silent.

(By contrast, the culture surrounding RPMs has generally been strongly in the 'no noise is good news' camp, where RPMs should not generate output during installation, removal, and upgrades unless something goes wrong.)

Sidebar: the particular error we saw

For the benefit of anyone who is curious, we saw error messages of the form:

Setting up linux-image-2.6.15-55-amd64-server (2.6.15-55.83) ...
ln: creating symbolic link `/tmp/mkinitramfs_8tcKVS/bin/true' to `/usr/lib/klibc/bin/true': File exists

I don't know what causes these. Since the machines did reboot into the new kernel and do not seem to have exploded, my motivation for digging into the depths of potential faults in initial ramdisk building is low.

Written on 25 March 2010.
« One reason why 'zpool status' can hang
Tkinter sometimes has a busy-wait main loop (more or less) »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Mar 25 01:14:33 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.