Thinking about our alternatives to Solaris

In light of yesterday's entry, the obvious question is what options we have for an alternative to Solaris in the next generation of our fileserver infrastructure. I've been thinking about this for a while, and at the moment there are three main alternatives.

The obvious option is OpenSolaris Illumos. However, there's a lot of question marks around whether Illumos will really be able to deliver a viable open source Solaris alternative. There's also even bigger question marks about whether it makes sense for us to build a core piece of infrastructure on top of an open source OS that we basically have to maintain ourselves. I say that we'll have to maintain itself because I don't expect an Illumos-based equivalent of Red Hat to appear, someone who provides a long-term supported stable version with security and functionality updates.

(There are people building storage products with Illumos, but we don't want a storage product, we want a distribution.)

Next up is FreeBSD with ZFS. This still suffers from many of the issues I've written about before, and my gut doesn't like it. However, I have to admit that if FreeBSD remains committed to ZFS it may be a better alternative than Illumos, since the FreeBSD team already has a solid track record for delivering a solid OS. And Oracle's decisions about OpenSolaris mean that the rate of ZFS changes to integrate into FreeBSD may slow way down.

The pessimistic flipside for both Illumos and FreeBSD is that right now we have no idea if the Oracle's public ZFS codebase will ever get any more updates; OpenSolaris may never get another public code drop. If it does not, both Illumos and FreeBSD would be left to reverse engineer and reimplement ZFS bugfixes and improvements, which might well not happen for one or both of them. This might well effectively fork ZFS, and I don't know if the open-source ZFS would really get developed further.

The final good alternative is Linux with btrfs, which promises the good features of ZFS but is currently entirely incapable of delivering them. Right now btrfs is is somewhere around half baked, with lots of good intentions, some amount of features, and a huge number of rough edges. Much of this will change in time if things go well, so I certainly hope that the picture looks rather different in two or three years, because there are no really good Linux-based alternatives other than btrfs.

Even in two or three years, building on btrfs will be risky. I'd expect that we'd be on the early edge of serious production deployments of it, so we might find all sorts of problems when using it at scale in an NFS fileserver environment.