Thinking about our alternatives to Solaris

March 16, 2011

In light of yesterday's entry, the obvious question is what options we have for an alternative to Solaris in the next generation of our fileserver infrastructure. I've been thinking about this for a while, and at the moment there are three main alternatives.

The obvious option is OpenSolaris Illumos. However, there's a lot of question marks around whether Illumos will really be able to deliver a viable open source Solaris alternative. There's also even bigger question marks about whether it makes sense for us to build a core piece of infrastructure on top of an open source OS that we basically have to maintain ourselves. I say that we'll have to maintain itself because I don't expect an Illumos-based equivalent of Red Hat to appear, someone who provides a long-term supported stable version with security and functionality updates.

(There are people building storage products with Illumos, but we don't want a storage product, we want a distribution.)

Next up is FreeBSD with ZFS. This still suffers from many of the issues I've written about before, and my gut doesn't like it. However, I have to admit that if FreeBSD remains committed to ZFS it may be a better alternative than Illumos, since the FreeBSD team already has a solid track record for delivering a solid OS. And Oracle's decisions about OpenSolaris mean that the rate of ZFS changes to integrate into FreeBSD may slow way down.

The pessimistic flipside for both Illumos and FreeBSD is that right now we have no idea if the Oracle's public ZFS codebase will ever get any more updates; OpenSolaris may never get another public code drop. If it does not, both Illumos and FreeBSD would be left to reverse engineer and reimplement ZFS bugfixes and improvements, which might well not happen for one or both of them. This might well effectively fork ZFS, and I don't know if the open-source ZFS would really get developed further.

The final good alternative is Linux with btrfs, which promises the good features of ZFS but is currently entirely incapable of delivering them. Right now btrfs is is somewhere around half baked, with lots of good intentions, some amount of features, and a huge number of rough edges. Much of this will change in time if things go well, so I certainly hope that the picture looks rather different in two or three years, because there are no really good Linux-based alternatives other than btrfs.

Even in two or three years, building on btrfs will be risky. I'd expect that we'd be on the early edge of serious production deployments of it, so we might find all sorts of problems when using it at scale in an NFS fileserver environment.


Comments on this page:

From 187.45.255.252 at 2011-03-16 10:00:29:

We've have come to the same conclusion and, for now, it makes sense to pay Oracle for ZFS. Or consider the possibility of buying a storage appliance from some other vendor.

I don't expect btrfs on Linux to be as stable as ZFS is right now.. not in the next 3-5 years.

By cks at 2011-03-16 13:18:15:

We pretty much don't have the budget for anything that costs substantially more than a do-it-yourself solution. It's possible that Oracle's Solaris hardware and support prices will be reasonable, but 'Oracle' and 'reasonable prices' are not things that are usually heard together. That would leave us with no choice but to do something, whatever it is.

From 77.249.14.105 at 2011-03-16 14:24:57:

and nexenta: http://www.nexenta.org/ ?

By cks at 2011-03-16 15:14:55:

It remains to be seen if Nexenta will be more like Fedora, more like Debian, or more like Red Hat Enterprise Linux or Ubuntu LTS in terms of support duration and stability. My current guess is that it will be more like Debian (to the extent that it is like anything at all).

From 69.43.155.250 at 2011-03-18 20:03:52:

Fortunately, UToronto has a very nice site license for NexentaStor :-)

From 66.92.52.243 at 2011-03-19 17:42:04:

The founder of illumos works for Nexenta. Joyent is throwing in code as well, and their new infra is illumos-based. The illumos infra is hosted (and managed by me) on Joyent and EveryCity zones. EveryCity's CEO started the OpenIndiana project, which will be illumos-based starting with its stable release.

bmc (co-creator of DTrace, and now VP at Joyent) is on record as saying "Fuck Solaris, illumos is where it's at."

Cheers.

-- bdha

From 91.178.253.133 at 2011-05-21 02:47:06:

Have you looked at other "corporate solutions" like IBM's GPFS, Quantum's StorNext or HP's ibkix? They provide multi-server storage solutions with a lot of features. We're currently looking into them at Brussel's University.

Thanks,

Michel

By cks at 2011-05-24 11:30:20:

We haven't looked at any vendor solutions, partly because we're not yet at the point where we feel we have to seriously evaluate alternatives to Solaris. Especially for vendor solutions, we could only sensibly evaluate them very close to the point where we'd buy them.

My gut feeling is that any vendor solution is likely to cost too much or be too inflexible, and probably both.

Written on 16 March 2011.
« Our uncertain future with Solaris 11
The timing of production Linux deployments »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Mar 16 04:04:01 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.