Our likely ZFS fileserver upgrade plans (as of March 2019)
Our third generation of ZFS fileservers are now in full production, although we're less than half way through migrating all of our filesystems from our second generation fileservers. As peculiar as it sounds, this makes me think ahead to what our likely upgrade plans are.
Our current generation ZFS fileservers are running Ubuntu 18.04 LTS with the Ubuntu version of ZFS (with a frozen kernel version). Given our past habits, it's unlikely that we'll want to upgrade them to Ubuntu 20.04 LTS when that comes out in a year or so, unless there's some important ZFS bugfix or feature that's present in 20.04 (which is possible, cf, although serious bugs will hopefully be fixed in the 18.04 version of ZFS). Instead, we'll only start looking at upgrades when 18.04 goes on its end of life countdown when Ubuntu 22.04 LTS comes out, which historically will be in April of 2022, three years from now.
In 2022, our current server hardware and 2TB data SSDs will be about four years old; based on our past habits, this will not be old enough that we consider them in urgent need of replacement. I hope that we'll turn over the SSDs for new ones with larger capacity (and without four years of write wear), but we might not do it in 2022 at the same time as we execute an upgrade to 22.04. If we have money, we might refresh the servers with new hardware, but if so I think we'd mostly be doing it to have hardware that hadn't been used for four years, instead of more powerful hardware, and in general our SuperMicro servers have been very reliable; our OmniOS generation are now somewhere around five years old and show no signs of problems anywhere. The one exception is that maybe RAM prices will finally have gone down substantially by 2022 so we can afford to put a lot more memory in a new generation of servers.
(We will definitely be upgrading from Ubuntu 18.04 when it starts going out of support, and it's probable that it will be to the current Ubuntu LTS instead of to, say, CentOS. Hardware upgrades are much more uncertain.)
Frankly, next time around I would like us not to have to move our ZFS pools and filesystems over to new fileservers; it takes a lot of work and a lot of time. An 'in place' upgrade for the ZFS pools is now at least possible and I hope that we do it, either by reusing the current servers and swapping in new system disks set up with Ubuntu 22.04, or by moving the data SSDs from one physical server to another and then re-importing the pools and so on.
(We did a 'swap the system disks' upgrade on our OmniOS fileservers when we moved from r151010 to r151014 and it went okay. It turns out that we also did this for a Solaris 10 upgrade many years ago.)