Wandering Thoughts archives

A brief introduction to ZFS (disk) GUIDs

Although ZFS commands like 'zpool status' will generally not tell you this, ZFS does not actually identify pool devices by their device name. Well, mostly it doesn't. The real situation is somewhat complex.

ZFS likes identifying pool-related objects with what it calls a 'GUID', a random and theoretically unique 64-bit identifier. Pools have GUIDs, vdevs have GUIDs, and, specifically, disks (in pool configurations) have GUIDs. ZFS internally uses the GUID for most operations; for instance, almost all of the kernel interfaces that zpool uses actually take GUIDs to identify what to change, instead of device names.

(The 'numeric identifier' that you can use to have 'zpool import' import a pool is the pool's GUID.)

In a pool's configuration, entries for disks have a bunch of information to help ZFS identify the right device: the GUID of the disk, the device it's expected to be found on, and the physical path and device ID of that device. You can see most of a pool's raw configuration, complete with this information about each disk, with 'zdb -C <pool>'.

(Unfortunately, zdb doesn't print information about spare disks, only about disks that are in vdevs.)

As you might guess, disks being used by ZFS have an on-disk label (in fact they have four copies of it, two at the start of the ZFS slice and two at the end). Among other things, this disk label has the disk's GUID. You can dump a disk's ZFS label with 'zdb -l' on the ZFS slice (normally slice 0, 's0', if you have given ZFS the full disk).

(On disks that are part of a live vdev, the disk label also has a copy of the vdev's information; on spare disks, all the label has is the disk's GUID, the version, and the state.)

Conceptually, 'zpool import <pool>' works by finding a copy of the nominal full pool configuration and then searching all of the disks to find the disk GUIDs mentioned in the pool configuration. If the system can find enough of the disks, it can actually import and bring up the pool.

(I'm not entirely clear where the full pool configuration is stored; it's in the pool somewhere, but it's unfortunately not in the disk labels, so it's not trivial to dump it with zdb.)

Note that ZFS GUIDs are not real GUIDs. Real GUIDs are 128-bit objects and are conventionally printed in a special format; ZFS GUIDs are only 64-bit ones and are conventionally printed as plain decimal numbers.

Sidebar: zdb versus the full pool configuration

What seems to be going on with 'zdb -C' is that it dumps out the in-kernel pool configuration nvlist, and while the kernel keeps track of spares and other things, it does not keep them in the in-kernel pool config nvlist; instead it stuffs them into other bits of data structures that zdb does not print out.

Things like 'zpool status' look at the full pool configuration, but they don't print the raw nvlist; instead they helpfully process it for you and hide various bits of what is going on.

(I wound up writing a full pool config nvlist dumper myself; you can get a copy of the current source code here.)