Every so often I poke at some aspect of our fileserver replacement project (where we need to replace our current Solaris 10 Update 8 servers with something modern enough to handle 4K sector disks), but at the moment things are moving slowly. One reason for this slowness is that I hope that things will get clearer as time goes on.

Currently I'm looking at Illumos and ZFS on Linux. With Illumos, I brought up most of our environment on an OmniOS VM and it all worked (including some tricky bits) and almost all of it worked just like Solaris. With ZFS on Linux I have the core basics up but I'm slowly chasing a NFS performance issue. And in a way this encapsulates the overall issue for me.

The risk with ZFS on Linux is issues integrating the ZFS codebase with Linux. My NFS write performance issue is clearly an issue at this join point and I have few concrete ideas for how to either troubleshoot it or to resolve it. It's probably not the only such integration issue out there and the only way to smoke them out (or to be sure that they aren't going to affect us) may be to run ZoL in production in our environment.

(I admit that that's the pessimistic view.)

The risk with Illumos is the same as it always has been: that we won't be able to find an Illumos distribution that is mature and supported for a long time, or at least not a distribution that has what we want. OmniOS has what we want and tracking it over time will tell me something about the other attributes. Not huge amounts, though, so I think I am going to have to start following some mailing lists so I can get an informed idea of how things are going.

(A project's mailing lists often give you a somewhat too pessimistic view of how healthy the project is because they often attract people with problems or gripes instead of all of the people who are happy. But seeing what the problems and gripes are is itself interesting, as is finding out what the explosive political issues are. It's just that mailing lists are time consuming and it's hard to sustain interest if you don't care about the problems, you're just there to get a sense of the land.)

In that our fileservers are going to be locked down appliances that we rarely update or even touch, my somewhat reluctant current belief is that any Illumos distribution is probably going to wind up less risky than ZFS on Linux. In practice we can have much more confidence in the core ZFS, NFS, iSCSI, and multipathing environment on Illumos because basically all of it comes from Solaris and we have plenty of experience with most of the Solaris bits. If the worst comes to the worst, lack of updates is not a huge drawback once we freeze the production system.