Why ZFS needs a zfsdump

February 6, 2008

ZFS needs a zfsdump program for the same reasons that every filesystem needs a *dump program: you need something that can fully and completely reproduce the state of the on-disk data, complete with sparse files and weird permissions and so on, and can restore small portions of the backup not just the whole thing.

(In this day of more and more people turning off atime, it's probably no longer so important to have a backup tool that is guaranteed not to change the state of files.)

Generalized backup tools like GNU tar can do partial restores, but cannot completely capture things like sparse files. zfs send and zfs receive can exactly capture sparse files and so on, but do not support partial restores.

(There are other significant drawbacks of zfs send as a backup mechanism, including the issue with snapshots and quotas if you want to do incremental backups. My strong impression is that the whole mechanism is only really intended for transferring filesystems between pools and replication.)

Unfortunately it looks like almost all of the real work of zfs send is done deep in the kernel ZFS code, so you can't reuse the user level stuff and just make it generate dump compatible output instead of the current stream format.


Comments on this page:

From 70.53.122.200 at 2008-02-07 08:42:50:

I've heard that "star" handles sparse and special files much better than other tar programs.

http://cdrecord.berlios.de/old/private/star.html

By cks at 2008-02-07 09:05:04:

The problem is that no portable program can handle sparse files correctly, because the Unix API gives you no way to know where the sparse portions are. You need to either use a private API that the kernel or filesystem exposes for this purpose, or read the information directly from the filesystem's data structures on disk.

From 128.100.48.224 at 2008-02-07 13:17:49:

According to star's website:

  • accurate sparse files

star is able to reproduce holes in sparse files accurately if the OS includes the needed support functions. This is currently true for Solaris-2.3 to Solaris-2.5.1 and for Solaris-10 or newer

so it should work fine on Solaris 10. The author, Joerg Schilling, has a lot of experience with Solaris.

OdR

From 128.100.48.224 at 2008-02-07 13:38:40:

you might need the alpha/beta versions of star 1.5 for Solaris 10/11 sparse file support though. The 1.4 releases seem to be pre-Solaris 10.

From 192.18.128.5 at 2008-11-21 13:23:56:

I think the star proponents are missing the point. We want to recreate the filesystem in it's entirety. For ZFS, this includes snapshots and clones. I doubt star will ever have this capability.

John Cecere

By cks at 2008-11-23 14:13:29:

I don't think that the issue is quite that clear-cut. Certainly for our situation here, the data in the filesystem is the important thing, and not so much the pool structure (snapshots, clones, etc). It would be nice to have a tool that backed up entire pools instead of just filesystem data, but I'd be happy with a 'zfsdump' that just did the latter.

Written on 06 February 2008.
« Prewiring experimental racks
Why /usr/local is now useless (and where it came from) »

Page tools: View Source, View Normal.
Search:
Login: Password:

Last modified: Wed Feb 6 23:01:29 2008
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.