We're starting to be vaguely in sight of the point at where we'll have to consider alternatives to our current fileservers. While the software environment hasn't changed, our fileserver infrastructure is built on SunFire X2200s and X2100s and we've already had two hardware failures in our X2100 pool. We have some number of spare X2200s and of course there's more modern hardware, but the problem is Solaris licensing; there's a strictly limited pool of spare X2200s that we can legally use with Solaris.

Oracle Solaris (on new hardware) is not an option. It costs too much (especially once you include likely hardware costs), we'd have no source code access (which has been very important to us), and most of all I have no trust in Oracle's long-term behavior. If we're building a new fileserver environment that's supposed to last at least five years, Oracle's handling of Solaris and Solaris licensing issues to date has created too much uncertainty for me.

The straightforward alternative version of 'Solaris' is one of the Illumos-based distributions. This raises the question of 'which one', which in turn (for me) raises the question of just what we want and need in the 'Solaris' that we use.

I'm afraid that my answer is going to be boring: we want a traditional Unix server OS. We want something with an /etc/passwd, cron, et al, something that we log in to through SSH and get a Bourne shell command line with vi and so on. We're going to continue to configure and manage ZFS pools our way and with our own tools (and we're going to continue to use iSCSI in a very specific way); we have no interest in web management interfaces, integrated NAS systems, and so on. I'd like to see better package management and more packages in the base system (it's 2012, everyone should be packaging rsync), but if I'm being honest it's not a very high priority because we're not going to update these systems unless we can't help it.

We absolutely need source code. Lack of source code almost certainly would rule an Illumos-derived distribution out of the running entirely. Among other reasons for source, DTrace is important and it's not very useful without source.

Support is not a priority. Having support nominally available is reassuring but in practice we've never gotten actual support from Sun and it's all but certain that source code and open source support channels (IRC, etc) will provide better results. Similarly, having the specific release of a distribution updated for a long time is also reassuring but probably not useful in practice; we're extremely likely to treat these machines as appliances and never apply updates to them.

(If nothing else, having no official support available will waste less time because I won't be spending a bunch of time talking to a support organization that doesn't actually solve my problems.)