Keeping backup ZFS on Linux kernel modules around

July 26, 2020

I'm a long term user of ZFS on Linux and over pretty much all of the time I've used it, I've built it from the latest development version. Generally this means I update my ZoL build at the same time as I update my Fedora kernel, since a ZoL update requires a kernel reboot anyway. This is a little bit daring, of course, although the ZoL development version has generally been quite solid (and this way I get the latest features and improvements long before I otherwise would).

One of the things I do to make it less alarming is that I always keep backup copies of previous versions of ZFS on Linux, in the form of copies of the RPMs I install and update. Naturally I keep these backup copies in a non-ZFS filesystem, because I need to be able to get at them even if the new version of ZFS isn't working (possibly just with the new kernel, possibly in general). I haven't needed these backup copies very often, but on the rare occasions when I've had to revert, I was very glad that they were there.

(You don't always run into immediate failures to bring ZFS up; sometimes there are merely stability or other issues in a new development change, and you want to roll back to a previous one. In those cases it's okay to have the previous versions on a ZFS filesystem, because you can probably use ZFS enough to grab them.)

Not everyone uses development versions of ZFS on Linux, but I suggest that you keep backup copies of older versions even if you only use released ZoL versions. You never know when you may run into an issue and be glad that you have options.

(That I keep backup copies of previous versions and want to have them accessible outside of ZFS is one reason that I doubt I'll ever use ZFS on Linux on my root filesystem. System recovery is much easier in many scenarios if ZFS isn't required to at least boot the system, get it on the network, or access the root filesystem from a live CD.)

One obvious requirement here is that you should never update ZFS pool or filesystem features until you're absolutely sure that you'll never want to revert to a ZoL version that's too old to support those features. This generally makes me quite conservative about updating pool features; I want them to be in a ZoL release that's been out long enough to be considered fully stable.

Comments on this page:

By Opk at 2020-07-26 16:55:18:

When you say that you're keeping backups of RPMs, are you generating RPMs for the actual compiled modules as opposed to just using DKMS? I've pondered creating such RPMs, partly because we're using ZFS on half a dozen workstations but have never got around to working out a procedure for building them. Otherwise, we sync all third party yum/dnf repositories including so could only revert if the old modules still build. The only times I've had issues, the workaround consisted of going back to an old kernel with whatever zfs that had.

Out of interest, would you use ZFS on Linux for root on Ubuntu assuming their provided kernels? I'd assume that's fine but don't use Ubuntu myself. I likewise always keep root on native Linux filesystems. And so miss the boot environments of FreeBSD and Solaris.

By cks at 2020-07-26 18:03:34:

I'm just using DKMS, so I keep backups of the zfs-dkms RPMs. This can go wrong if an update breaks DKMS instead of ZFS itself (which happens every so often), but usually I notice before I reboot into the new kernel, because I check with 'dkms status' to make sure the modules are ready.

I'm not sure if I'd use ZFS on Linux for the root filesystem even with Ubuntu. Certainly I wouldn't if their live CDs don't have it there, but even if the live CDs support it I might be a bit wary. I also don't see ZFS root as terribly compelling for the stuff we do, especially with our Ubuntu servers.

I also don't see ZFS root as terribly compelling for the stuff we do, especially with our Ubuntu servers.

Where ZFS-root (on any operating system) would be useful is with boot environments: create a ZFS snapshot of your system, make a writable clone, install updates there, and reboot into it. If things don't work then boot back into the pre-update environment.

The idea came out first on Solaris, but FreeBSD has it now as well. The Ubuntu folks are working on something with zsys:

Written on 26 July 2020.
« My varied types of Firefox windows
Linux PAM leads to terrible error messages from things like passwd »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Jul 26 00:22:10 2020
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.