== Why I like ZFS's _zfs send_ and _zfs receive_

Our new fileserver environment to be has reached the point where it
needs real testing, so on Monday I took the big leap and moved my home
directory filesystem to our first new fileserver in order to give it
some real world usage. Actually that's a bit of a misnomer. I'd copied
my home directory filesystem over to the new fileserver several weeks
ago simply to put some real data into the system; what I did on Monday
was re-synchronize the copy with the live version and then switch all of
our actual Ubuntu servers to NFS mounting it from the new fileserver.

This is exhibit one of why I like _zfs send_ and _zfs receive_, or more
specifically why I like incremental _zfs send_ and _zfs receive_.  They
make it both easy and efficient to copy, repeatedly update, and then
synchronize filesystems like this during a move. You can sort of do the
same thing with _rsync_ at the user level but _zfs send_ does it faster
(sometimes drastically so), to some degree more reliably, and certainly
more conveniently.

(As I've found the hard way, if there is enough of the wrong sort of
activity in a filesystem an incremental _rsync_ can take very close to
the same amount of time as a non-incremental one. That was a painful
experience. This doesn't happen with _zfs send_; you only ever pay for
the amount of data actually changed.)

The other reason why I'm so fond of _zfs send_ is this magic trick:
~~incremental _zfs send_ with snapshots is symmetric~~. Once you've
synchronized two copies this way you can do it in either direction and
you can reverse directions partway through.  My home directory's move
is almost certainly temporary (and it's certainly temporary if problems
arise) and as long as I retain the filesystems and snapshots on both
sides I can move *back* just as fast as I moved in the first place. I
don't have to make a full copy and then synchronize it and so on; I
can just make a new snapshot on the new fileserver and send back the
difference between my last 'move over' snapshot and this first 'move
back' snapshot. Speaking as someone who's currently basically jumping up
and down on a new fileserver that should be good but hasn't been fully
proven yet, that's a rather reassuring thing.

(In fact if I'm cautious I can update my old home directory every
night or just periodically, so that I won't lose much even if the new
environment goes down in total flames somehow.)

PS: Since I've now tested this in both directions, I can say that
you can _zfs send_ a filesystem from Solaris 10 update 8 to OmniOS
and then _zfs send_ snapshots from it from OmniOS back to S10U8
(provided that you don't '_zfs upgrade_' the filesystem while it's
on OmniOS; if you do, it'll become incompatible with S10U8's old
ZFS version (or at least so the documentation says, I haven't tested
this personally for the obvious reasons)).

=== Sidebar: how we freeze filesystems when moving them

The magic sequence we use is to unshare the filesystem (since all
of our access to filesystems is over NFS), set _readonly=on_, and
then make the final move snapshot. After the newly moved filesystem
is up and running we set _canmount=off_ on the old version and then
let it sit until we have a backup of the moved one and everything
is known to be good and so on.

We almost always do _zfs receive_ into unmounted filesystems (ie
'_zfs receive -u ..._'.)