Wandering Thoughts archives

Why I believe that you want Solaris if you want ZFS

As Wolfgang Lonien recently pointed out, Solaris isn't the only system with ZFS, as you can also get it in at least FreeBSD. Despite these ports, I believe that Solaris generally remains your only solid choice if you need ZFS.

The problem is that ZFS is currently incomplete, with both missing features and significant problems. This means that the work of integrating ZFS into other systems is not complete, and there are two potential issues with the remaining work.

The first is the most straightforward: you are taking a risk that Sun will change the license terms of future, more complete versions of ZFS in a way that leaves the code not usable by your operating system. This would maroon you with the current incomplete ZFS.

(I don't believe that this is a big risk, but I am not convinced that Sun really gets open source in its heart, and I am pretty convinced that they see ZFS as a core weapon in their commercial strategy.)

The second is that what operating systems will need to put in work in the future to track and merge Sun's upstream changes. This is at least time consuming, delaying how fast fixes and features reach you, and you run the risk that it simply won't get done, that the necessary developers will run out of interest.

(My impression is that integrating upstream changes is a tedious, unglamorous job at the best of times. And I'm not sure that any OS will really commit itself to ZFS given the various uncertainties around it, which lowers maintainer motivation.)

Or in short: the current state of ZFS is such that other operating systems can't really just fork the codebase, ignore Sun, and go their own way. Keeping up with Sun will take effort and time and means that Sun could in theory cut people off, so what you see now in other OSes may be all you'll really get.

The robot logic of ZFS snapshots and quotas

One of the things that irritate me about ZFS is the interaction between snapshots and quotas, specifically that ZFS snapshots count against a filesystem's quota. I can only call this robot logic; it is in a sense completely correct and it probably simplifies the implementation, but it is not useful.

The problem is that snapshots are a system level thing; having them count against user quotas is a problem waiting to happen. Consider trying to explain to a user that they have 'run out' of disk space because the system people have made a snapshot to run backups, which is just the situation you might get with the current ZFS approach. Not counting snapshots against quotas doesn't let users use any more space than they should, since they can't make snapshots themselves. Systems people making snapshots can 'exceed' the quota, but then they could just raise the quota directly.

(The possibility for user confusion is huge; consider, for example, that deleting a big file won't reduce your disk space usage when there is a snapshot hanging around.)

I am relatively twitchy about this issue partly because around here it's routine for some filesystems to be mostly or entirely full, so it's clear that we could easily run into this issue fairly frequently. If your quotas are just a precaution against runaway programs accidentally writing huge data files or the like, you will probably care much less.

Some thoughts on Solaris 10 x86 versus Linux

Someone around here recently asked me my opinions on Solaris on x86 machines, and in thinking about it I decided I might as well put down my current thoughts on the whole matter.

First, we're no longer interested in SPARC hardware and thus SPARC Solaris. The hardware is clearly no longer competitive for generic computing; it may be competitive in some niche roles, but my area doesn't have any of those niches we need filled.

(In the long run I feel that a retreat to specialized areas dooms SPARC in general, for reasons discussed here.)

In the old days, x86 Solaris was Sun's red headed stepchild, which left some of us nervous when Sun decided to wake up. The good news is that Sun does seem to understand that x86 is most of their future; I'm convinced about Sun's commitment to Solaris 10 x86 in general (Sun's peculiar desire to recreate OpenBoot notwithstanding).

I am running Solaris 10 x86 on test machines, but we haven't yet put it into production. So far my experience is biased because I have been torturing the machines (and running into a serious issue that Sun was slow to deal with), so I'm not going to try to draw general conclusions. However, I believe that it will be solid in routine production use, and certainly there are people who are happy with it.

However, in general I still recommend Linux over Solaris 10 x86, for a number of reasons:

• the sysadmin experience on a modern Linux is much better than on Solaris 10. There really is no comparison between, say, apt-get and the official Solaris patch management tools.

• there is much more useful software conveniently available for your average Linux distribution (either as prepackaged binaries or as easy to build source code).

• Solaris's driver support seems lacking. Some things (eg, jumbo frame capable Ethernet drivers) are backwards compared to Linux; others that I would like are just missing, such as drivers for 3Ware Escalade controllers.

  (Note: 'supported in Nevada' doesn't count, except as an indication that it might show up in Solaris 10 at some indeterminate time.)

• I don't know how well Solaris 10 x86 runs on non-Sun x86 hardware, especially recent hardware.

  (Of course, it runs fine on Sun x86 hardware and Sun has competitively priced x86 stuff these days, so this may not matter to you.)

The ease of Linux administration and the Linux software selection are probably only an important advantage for people starting from scratch; if you already have Solaris machines and just want more, Solaris 10 x86 should do fine.

While Solaris 10 has technological advantages, I don't think that they are compelling enough to overcome the other issues for general purpose servers. Of course, if you need ZFS there's only one place to get it.