How you migrate ZFS filesystems matters
If you want to move a ZFS filesystem around from one host to another,
you have two general approaches; you can use '
zfs send' and '
receive', or you can use a user level copying tool such as
tar -cf | tar -xf', or any number of similar options). Until
recently, I had considered these two approaches to be more or less
equivalent apart from their convenience and speed (which generally
tilted in favour of '
zfs send'). It turns out that this is not
necessarily the case and there are situations where you will want
one instead of the other.
We have had two generations of ZFS fileservers so far, the Solaris
ones and the OmniOS ones.
When we moved from the first generation to the second generation,
we migrated filesystems across using '
zfs send', including the
filesystem with my home directory in it (we did this for various
reasons). Recently I discovered
that some old things in my filesystem didn't have file type
information in their directory entries. ZFS
has been adding file type information to directories for a long
time, but not quite as long as my home directory has been on ZFS.
This illustrates an important difference between the '
approach and the
rsync approach, which is that
zfs send doesn't
update or change at least some ZFS on-disk data structures, in
the way that re-writing them from scratch from user level does.
There are both positives and negatives to this, and a certain amount
of rewriting does happen even in the '
zfs send' case (for example,
all of the block pointers get changed, and ZFS
will re-compress your data as applicable).
I knew that in theory you had to copy things at the user level if
you wanted to make sure that your ZFS filesystem and everything in
it was fully up to date with the latest ZFS features. But I didn't
expect to hit a situation where it mattered in practice until, well,
I did. Now I suspect that old files on our old filesystems may be
partially missing a number of things, and I'm wondering how much
of the various changes in '
zfs upgrade -v' apply even to old data.
(I'd run into this sort of general thing before when I looked into ext3 to ext4 conversion on Linux.)
With all that said, I doubt this will change our plans for migrating our ZFS filesystems in the future (to our third generation fileservers). ZFS sending and receiving is just too convenient, too fast and too reliable to give up. Rsync isn't bad, but it's not the same, and so we only use it when we have to (when we're moving only some of the people in a filesystem instead of all of them, for example).
PS: I was going to try to say something about what '
zfs send' did
and didn't update, but having looked briefly at the code I've
concluded that I need to do more research before running my keyboard
off. In the mean time, you can read the OpenZFS wiki page on ZFS
send and receive,
which has plenty of juicy technical details.
PPS: Since eliminating all-zero blocks is a form of compression, you can turn zero-filled files into sparse files through a ZFS send/receive if the destination has compression enabled. As far as I know, genuine sparse files on the source will stay sparse through a ZFS send/receive even if they're sent to a destination with compression off.