Wandering Thoughts archives

The temptation to rebuild my office machine with its data in ZFS on Linux

A while back, I added a scratch drive to my office workstation to have more space for relatively unimportant data, primarily ISOs for virtual machine installs and low priority virtual machine images and extra disks (which I used to test, eg, virtual fileservers and iSCSI backends). About a month ago I gave into temptation and rebuilt that disk space as a ZFS pool using ZFS on Linux, fundamentally because I've come to really like ZFS in general and I really wanted to try out ZFS on Linux for something real. The whole experience has gone well so far; it wasn't particularly difficult to build DKMS based RPMs from the development repositories, everything has worked fine, and even upgrading kernels has been completely transparent to date.

(The advantage of building for DKMS is that the ZFS modules get automatically rebuilt when I install Fedora kernel upgrades.)

All of this smooth sailing has been steadily ratcheting up the temptation to go whole hog on ZFS on Linux by rebuilding my machine to have all of my user data in a ZFS pool instead of in the current 'filesystems on LVM on software RAID-1' setup (I'd leave the root filesystem and swap as they are now). One of the reasons this is extra tempting is that I actually have an easy path to it because my office workstation only has two main disks (and can easily fit another two temporarily).

I've been thinking about this for a while and so I've come up with a bunch of reasons why this is not quite as crazy as it sounds:

• given that we have plenty of spare disks and I can put an extra two in my case temporarily, moving to ZFS is basically only a matter of copying a lot of data to new disks. It would be tedious but not particularly adventurous.

• I have automated daily backups of almost all of my machine (and I could make that all of my machine) and I don't keep anything really crucial on it, like say my email (all of that lives on our servers). If ZFS totally implodes and eats my data on the machine, I can restore from them.

• I can simply save my current pair of disks to have a fairly fast and easy way to migrate back from ZFS. If it doesn't work out, I can just stick them back in the machine and basically rsync my changes back to my original filesystems. Again, tedious but not adventurous.

• My machine has 16GB of RAM and I mostly don't use much of it, which means that I think I'm relatively unlikely to run into what I consider the major potential problem with ZoL. And if I'm wrong, well, migrating back will merely be tedious instead of actively painful.

Set against all of this 'not as crazy as it looks' is that this would be daring and there wouldn't be any greater reason beyond just having my data in ZFS on my local machine. We're extremely unlikely to have any use for ZoL in production for, well, a long time into the future.

(And I probably wouldn't be building up enough confidence with ZoL to do this to my home machine, because because most of these reasons don't apply there. I have no spare data disks, no daily backups, and no space for extra drives in my case at home. On the attractive side, a ZFS L2ARC would neatly solve my SSD dilemma and my case does have room for one.)

One of the reasons that this is quite daring is the possibility that at some point I wouldn't be able to apply Fedora kernel updates or upgrade to a new Fedora version because ZoL doesn't support the newer kernels yet. Since I also use VMWare I already sort of live with this possibility today, but VMWare would be a lot easier to do without for some noticeable amount of time than, well, all of my data.

(The real answer is that that sort of thing would force me to migrate back to my current setup, either with the rsync approach or just by throwing another two new disks in and copying things from scratch.)

Still, the temptation is there and it keeps getting stronger the longer that my current ZoL pool is trouble free (and every time I run 'zpool scrub' on it and admire the assurance of no errors being detected).

(Having written this, I suspect that I'll give in in a week or two.)

PS: I don't think I currently have any software that requires filesystem features that ZoL doesn't currently implement, like POSIX ACLs. If I'm wrong about that a move to ZFS would be an immediate failure. I admit that I deliberately put a VMWare virtual machine into my current ZFS pool because I half expected VMWare to choke on ZFS (it hasn't so far).

PPS: I have no particular interest in going all in on ZoL and putting my root filesystem into ZFS, for various reasons. I like having the root filesystem in a really simple setup that needs as few moving parts as possible, so today it's not even in LVM, just on a software RAID-1 partition (the same is true for swap). Flexible space management is really overkill because the size of modern disks means that it's easy to give the root filesystem way more space than it will ever need. I do crazy things like save a copy of every RPM I've ever installed or updated through yum and my work root filesystem (/var included) is still under 60 GB.