The irritation of all of the Ubuntu kernels you wind up with

October 7, 2015

Let's start with my tweet:

I really dislike just how difficult Debian and Ubuntu make it to only keep the last N kernels and remove all the rest. What a stupid mess.

(I may be unfairly slamming Debian here, but if so they have their own terrible problem with kernel updates.)

One of the many stupid things about the Ubuntu kernel update process is how if you just use 'apt-get install' to install kernel updates, you'll wind up with a steadily increasing collection of old kernels. This isn't because Debian and Ubuntu care deeply about never removing a good kernel out from underneath you, as Ubuntu will happily overwrite a good kernel with a bad one sometimes (and Debian may be worse here). Instead, as far as I can tell, it is just because APT doesn't support this and no one has fixed it in more than a decade.

This matters for reasons beyond disk space and clutter in your list of installed package. Dpkg kernel updates are already kind of slow and definitely verbose enough that you can miss important things, and every installed kernel you have adds its own contribution to both the slowness and the verbosity. The fewer installed kernels you have, the faster things update and the higher the chance is that you'll notice any problems.

As they say, but wait, it gets worse. Not only does apt not support limiting how many kernels it keeps around, but Ubuntu (and Debian) don't even ship with an add-on command to remove such surplus kernels for you. This is asinine. Essentially everyone is going to want to do this, it is something that is surprisingly tricky to get right (and easy to get wrong in dangerous ways), and the best that Ubuntu has to offer is Stack Overflow answers full of arcane (and incomplete) command line incantations, people's homegrown scripts, and recommendations of packages with pages of new dependencies on normal systems.

Since cleaning up this mess would be far too much work, our systems totter along with an increasing collection of totally useless and pointless kernels (most of them with serious security holes, since the existence of serious holes is usually what prompts us to upgrade kernels). I rather enjoy when we get to reinstall machines, because it means starting from scratch with a clean and very short list of kernels.

(I've written about this in quieter tones, for example in How Ubuntu and Fedora each do kernel packages. That entry also discusses why it happens this way; see the comments for some additional hair-raising details.)


Comments on this page:

By andyb at 2015-10-07 07:03:47:

is `purge-old-kernels` a utility that came to your attention? in trusty+ it's in the "bikeshed" pkg (no Depends, only Recommends). removes all but the running and last 2 kernel pkgs.

has been working nicely for me.

http://packages.ubuntu.com/trusty/bikeshed

By Ben Hutchings at 2015-10-07 07:08:32:

It used to be that APT had a blacklist to prevent 'apt-get autoremove' from removing any kernel packages. Now it keeps a list of specific kernel packages not to be autoremoved, including the latest installed and the two newest by version number. The old packages still won't be removed totally automatically, but 'apt-get autoremove' should do so. This change was included in Debian 8; I'm not sure which Ubuntu versions have it.

By cks at 2015-10-07 14:52:50:

On our Ubuntu machines, 'apt-get install bikeshed' says it would drag in a huge list of other packages. It looks like it wants to install all of the 'recommends' packages (and then their dependencies and so on). Looking it up we can manually override this, but that's not a good answer for us as we have a scripted machine setup procedure that would have to be customized to (re)install bikeshed correctly. It's not worth having to modify our machine setup system for this.

For whatever reason, 'apt-get autoremove' removes at most a handful of the large list of kernels. This may be an Ubuntu issue or it may be an artifact of the issue that we hold all kernel packages in dpkg and then install new kernels with 'apt-get install ...' to update the meta-packages and drag in their new dependencies.

Is there some reason --no-install-recommends won’t work for you…? (Maybe you consider that a manual override for some reason?)

By cks at 2015-10-07 21:38:44:

Yeah, that's the manual override. We'd have to change our install system to install one special package with --no-install-recommends.

By Ewen McNeill at 2015-10-11 18:31:43:

FWIW, I've found that since about Ubuntu Linux 13.10 (ie, definitely including Trusty/14.04 LTS) and since about Debian 7, if you "follow the rules", then "apt-get --purge autoremove" will remove "unnecessary" kernels. Where "follow the rules" means (a) don't explicitly install individual kernel packages (as that will mark them as required-by-admin rather than just required-by-dependency, and thus prevent auto-remove), and (b) don't put the kernel packages on hold (as IIRC auto-remove will not touch things that are on hold), which boils down to (c) install the appropriate kernel metapackage, and new kernels only as an automatic dependency of that metapackage getting updated. (Earlier versions, including Ubuntu 10.04 and 12.04 did not autoremove kernels safely.)

I've also successfully used a python script and a shell script that I wrote to manage kernels on multiple Ubuntu versions. The python script figures out what to remove, and the shell script initiates the removal. It tries to keep the running kernel, the most recent kernel, and one older one, and then remove the rest (whether or not they were admin-installed, or dependency-installed -- so I could clean up systems where the dependency-installed bits had been overridden).

Ewen

Written on 07 October 2015.
« How many recent sender domains are in the Spamhaus DBL
Why you (probably) want to have blog categories (and topics and more) »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Oct 7 01:58:53 2015
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.