In a comment on my last entry, Matt Simmons asked why you'd let ZFS handle RAID level issues instead of just handling them in your storage backend (SAN or otherwise). Having ZFS do this is the recommended practice, for a number of reasons (some of which the ZFS FAQs will tell you about).

Here's the reasons for letting ZFS handle the RAID stuff that I know about (or at least can think of right now):

• ZFS can reliably repair corrupted blocks in mirrors, because it can tell which copy of a block is bad and which is good.
• ZFS should be able to recover better from subtle RAID-5/6 corruption. A backend RAID setup can fix an outright read error just as well as ZFS can, but I believe that ZFS can also fix things if a sector gets corrupted instead of simply reporting read errors.

• ZFS reconstructs things faster and better after a disk failure than a SAN backend can.

• ZFS will probably run faster if it's scheduling IO relatively close to the physical disks. This especially applies if the physical disks are actually shared between pools behind ZFS's back (as is typical if you define a big RAID-5 or RAID-6 array and then carve it up into LUNs, each of which you turn into a separate pool).

• some write loads on RAID-5 or RAID-6 may run faster.

• without its own redundancy (across different backends), ZFS will panic your system if one of your backends is temporarily unavailable for more than a relatively short amount of time.

Having ZFS handle RAID level issues has some drawbacks, though. They includes at least:

• ZFS's versions of RAID-5 and RAID-6 have significantly lower random read rates.

• You have to count on ZFS to get spares right.

• various things hang if some of your disks go away in the wrong way, even though your pools are still working.

• ZFS is much less prepared for long term storage management than serious (and seriously expensive) 'enterprise' SAN systems, many of which can do really quite sophisticated rearrangements of your SAN arrays.

I would hope that storage backends have better status monitoring and reporting tools than ZFS currently does (because ZFS's are fairly bad), but I am not at all sure that that is the case.