My opinions on when you should let ZFS handle RAID stuff

Given the previous entry, here's my opinions so far on when you should let ZFS handle your redundancy and when you should have your storage backend do RAID. (This assumes that you have a storage backend; if not, well, you don't have much choice.)

• if you are doing mirroring, you really want to let ZFS handle it because ZFS will do a much better job of handling small problems than your storage backend can.

• if you have a SAN and need your frontends to survive temporary glitches in a single backend unit, you still need to use cross-backend mirroring with ZFS handling it.

  (This is our situation.)

• if you have a heavy random read worklog and can only afford the space overheads of RAID-5 or RAID-6, you pretty much have to let your backend handle the RAID stuff in order to keep your read rate up. You can play games with multiple smaller raidz or raidz2 vdevs, but you start losing more and more space to parity overheads and I'm not sure you get all that much for it.

(Thinking about it, with raidz you effectively add a disk's IOPs for every extra disk that you lose to parity overhead. Each vdev can do IO independently but only gives you one disk's IOPs, and costs you one additional disk of overhead.)

If you are seriously worried about data integrity, you probably want to let ZFS handle the RAID stuff unless you have lots of spare disk bandwidth. Otherwise you will be scanning all of your disks twice, once in the storage backend to verify the RAID array and once in ZFS to guard against all of the other things that can go wrong.

In general (as the title of the previous entry says) I think that you are better off having ZFS handle the redundancy unless there are strong reasons otherwise. Roughly speaking, how much better off depends on how much you spent on your storage backend; the cheaper the backend, the less that you want it doing RAID or indeed much of anything at all except getting out of your way.

(This is true in general, of course.)