The three different names ZFS stores for each vdev disk (on Illumos)
I sort of mentioned yesterday that ZFS keeps
information on several different ways of identifying disks in pools.
To be specific, it keeps three different names or ways of identifying
each disk. You can see this with '
zdb -C' on a pool, so here's
a representative sample:
# zdb -C rpool
MOS Configuration: [...] children: type: 'disk' id: 0 guid: 15557853432972548123 path: '/dev/dsk/c3t0d0s0' devid: 'id1,sd@SATA_____INTEL_SSDSC2BB08__BTWL4114016X080KGN/a' phys_path: '/pci@0,0/pci15d9,714@1f,2/disk@0,0:a' [...]
guid is ZFS's internal identifier for the disk,
and is stored on the disk itself as part of the disk label. Since
you have to find the disk to read it, it's not something that ZFS
uses to find disks, although it is part of verifying that ZFS has
found the right one. The three actual names for the disk are reported
devid aka 'device id', and
path is straightforward; it's the filesystem path to the
device, which here is a conventional OmniOS (Illumos, Solaris)
cNtNdNsN name typical of a plain, non-multipathed disk. As this
is a directly attached SATA disk, the
phys_path shows us the
PCI information about the controller for the disk in the form of
a PCI device name. If we pulled this
disk and replaced it with a new one, both of those would stay the
same, since with a directly attached disk they're based on physical
topology and neither has changed. However, the
devid is clearly
based on the disks's identity information; it has the vendor name,
the 'product id', and the serial number (as returned by the disk
itself in response to SATA inquiry commands). This will be the same
more or less regardless of where the disk is connected to the system,
so ZFS (and anything else) can find the disk wherever it is.
(I believe that the '
id1,sd@' portion of the
devid is simply
giving us a namespace for the rest of it. See '
prtconf -v' for
another representation of all of this information and much more.)
Multipathed disks (such as the iSCSI disks on our fileservers) look somewhat different. For them, the
filesystem device name (and thus
path) looks like
identifier>d0s0, the physical path is
, and the devid_ is not particularly useful in finding
the specific physical disk because our iSCSI targets generate
synthetic disk 'serial numbers' based on their slot position (and
the target's hostname, which at least lets me see which target a
particular OmniOS-level multipathed disk is supposed to be coming
from). As it happens, I already know that OmniOS multipathing
identifies disks only by their device ids,
so all three names are functionally the same thing, just expressed
in different forms.
If you remove a disk entirely, all three of these names go away for both plain directly attached disks and multipath disks. If you replace a plain disk with a new or different one, the filesystem path and physical path will normally still work but the devid of the old disk is gone; ZFS can open the disk but will report that it has a missing or corrupt label. If you replace a multipathed disk with a new one and the true disk serial number is visible to OmniOS, all of the old names go away since they're all (partly) based on the disk's serial number, and ZFS will report the disk as missing entirely (often simply reporting it by GUID).
Sidebar: Which disk name ZFS uses when bringing up a pool
Which name or form of device identification ZFS uses is a bit
complicated. To simplify a complicated situation (see
as best I can, the normal sequence is that ZFS starts out by trying
the filesystem path but verifying the devid. If this fails, it tries
the devid, the physical path, and finally the filesystem path again
(but without verifying the devid this time).
Since ZFS verifies the disk label's GUID and other details after opening the disk, there is no risk that finding a random disk this way (for example by the physical path) will confuse ZFS. It'll just cause ZFS to report things like 'missing or corrupt disk label' instead of 'missing device'.