Wandering Thoughts archives


ZFS changes filesystem device numbers on reboot

Here is an unpleasant discovery that we just finished making: in at least Solaris 10 U6, ZFS changes the device numbers of all of your filesystems when your system reboots. More specifically (and even worse), the device numbers of a pool's filesystems change whenever the pool is imported, including the implicit import that happens when a system boots up.

(This raises interesting questions of how NFS filehandles keep working across system reboots, which they do. Presumably they have some other stable identifier for a given ZFS filesystem that still fits in the NFS filehandle size limits.)

So, you might ask, what difference does this make? Unfortunately for ZFS (and people using it), some programs still count on device numbers being stable so that they can use the device number plus inode as a long term identifier for a particular file. One such program is GNU tar, which uses this assumption for its incremental backups. If the assumption is violated, GNU tar's incremental backups turn into complete 'level 0' backups, since they think that everything is new because no device numbers match.

GNU tar, as it happens, is at the heart of our backup system. (Not that you have much choice with ZFS, since it has no dump equivalent and no, zfs send doesn't count.)

So, if you have ZFS machines and a backup system that might possibly assume that device numbers are stable, be careful. If you haven't tested rebooting a machine with enough backed-up data that you can tell if you're suddenly doing a full backup instead of an incremental one, well, you might want to plan such a test.

(For GNU tar specifically it is possible to fix the situation by directly modifying GNU tar's index files, which are fortunately stored in plain text format. How to do so doesn't fit within the margins of this entry.)

solaris/ZFSVariableDeviceNumbers written at 00:39:10; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.