Some gotchas with ZFS in Solaris 10 update 6

Apart from the outright issue with the ZFS failmode setting, here is some things that I've run across during our testing and usage of ZFS on Solaris 10 update 6:

• 'zpool status -x' is of the opinion that a pool using an older on-disk format is an error, or at least something that should show up in its output. It is completely mistaken and this mistake makes it far less useful than it could be, since it now reports all of our pools all of the time, regardless of whether real errors have happened or not.

• while it seems that 'zpool upgrade' is transparent when the system is in use, 'zfs upgrade' does not seem to be; in my one test so far, active NFS clients got IO errors during it.

  (I did not test what local programs would see, since it's not relevant for our environment.)

The new ZFS feature of pools having host ownership works, even with un-upgraded pools (which is useful for us), but is a bit peculiar. The important feature is that if a fileserver goes down, its pools are imported on a different machine, and the fileserver reboots, the fileserver will no longer automatically import the pools (and thus destroy them). This makes failover scenarios much safer.

However, it seems that Solaris will no longer import pools at all on boot if they were last used on another machine, even if the pools have been released with 'zpool export' (and it gives no boot-time error messages about this). This might seem like an obscure scenario, but consider something like a power supply failure; what we'd like to do is import the pools on a backup server, fix the primary server's power supply so that we can boot it, export the pools on the backup server, and power up and boot the primary server and have it just go. Instead, we'll have to remember to manually import all of the pools (slowly).

(In writing this it occurs to me to wonder if 'zpool import -a' on the primary machine would actually work. I suspect not, but it's worth testing.)