2017-03-27
We're probably going to upgrade our OmniOS servers by reinstalling them
We're currently running OmniOS r151014 on our fileservers, which is the current long term support release (although we're behind on updates, because we avoid them for stability reasons). However, per the OmniOS release cycle, there's a new LTS release coming this summer and about six to nine months later, our current r151014 version will stop being supported at all. Despite what I wrote not quite a year ago about how we might not upgrade at all, we seem to be broadly in support of the idea of upgrading when the next LTS release is out in order to retain at least the option of applying updates for security issues and so on.
This raises the question of how we do it, because there are two possible options; we could reinstall (what we did the last time around), or upgrade the existing systems through the normal process with a new boot environment. Having thought about it, I think that I'm likely to argue for upgrading via full reinstalls (on new system disks). There's two reasons for this, one specific to this particular version change and one more general one.
The specific issue is that OmniOS is in the process of transitioning to a new bootloader; they're moving from an old version of Grub to a version of the BSD bootloader (which OmniOS calls the 'BSD Loader'). While it's apparently going to be possible to stick with Grub or switch bootloaders over the transition, the current OmniOS Bloody directions make this sound pretty intricate. Installing a new OmniOS from scratch on new disks seems to be the cleanest and best way to get the new bootloader for the new OmniOS while preserving Grub for the old OmniOS (on the old disks).
The more broader issue is that reinstalling from scratch on new disks every time is more certain for rollbacks (since we can keep the old disks) and means that any hypothetical future systems we install wind up the same as the current ones without making us go through extra work. If we did in-place upgrades, to get identical new installs we would actually have to install r151014 then immediately upgrade it to the new LTS. If we just installed the new LTS, there are various sorts of subtle differences and incompatibilities that could sneak in.
(This is of course not specific to OmniOS. It's very hard to make sure that upgraded systems are exactly the same as newly installed systems, especially if you've upgraded the systems over significant version boundaries.)
I like the idea of upgrading between OmniOS versions using boot environments in theory (partly because it's neat if it works), it would probably be faster and less of a hassle, and I may yet change my mind here. But I suspect that we're going to do it the tedious way just because it's easier on us in the long run.