Some rough things about the naming of SAS drives on Linux
This isn't a comprehensive look at the names of SAS drives because SAS (as a technology) has a lot of options. Instead this is a look at how directly attached SATA drives behind SAS controllers get named. I assume that real SAS drives get named the same (but I don't have any to test with) and I have no idea how things look for SAS (or SATA) drives behind a SAS expander or more complex SAS topologies. Also some of this depends on the SAS controller and its driver; we're using LSI cards driven by the mpt2sas driver.
To start with, SAS drives have sdX
names and kernel 'SCSI host names'
just like SATA drives do. As with SATA drives, sdX
drive names are
not necessarily stable across hotswaps. Unlike SATA drives, the SCSI
host names are not necessarily fully stable either; I've seen them
get renumbered within a SCSI host after a hotswap, so that what was
'scsi6:0:3:0
' becomes 'scsi6:0:8:0
'. So far, LSI cards have a single
SCSI host regardless of how many SAS ports they support and how those
ports are physically connected (eg with a 4x SFF-8087 multiplexer or
broken out as individual SAS ports).
(For background, see my basic overview of SAS and the names of Linux SATA devices, which covers kernel SCSI host names in more detail along with the three other device namespaces.)
All of this tells you nothing about the drive's physical slot. In SAS (or at least in mpt2sas for directly attached devices), physical location information is the SAS PHY number. To find this you must go grubbing around in sysfs, so I will just show you:
set -P # in bash cd /sys/block/sdX/device cd ../../.. ls phy-*
(There is probably a simpler path from the block device, but sysfs is the kind of thing where I find a working way and then stop.)
The PHY name will be of the form 'phy-6:0
', meaning PHY 0 on SCSI host
6. PHY numbers sometimes also show up in kernel messages, such as:
scsi 6:0:2:0: SATA: handle(0x000a), sas_addr(0x4433221101000000), phy(1), device_name(0x50014ee25df6de01)port-6:7: remove: sas_addr(0x4433221103000000), phy(3)
Mapping PHY numbers to actual physical hardware slots is something that you'll have to do yourself for your specific hardware. Please don't assume that PHY numbering matches how the card BIOS does it (as seen in BIOS information printouts during boot or if you go into the BIOS itself); for our LSI cards, it does not.
(Although it may be obvious, PHY numbers are reused between different SAS controllers. Several controllers may all have a PHY 0.)
Since the SCSI host name of a SAS drive in a given physical slot
is not stable, /dev/disk/by-path
sensibly does not use it for SAS
drives. Instead it uses the 'SAS address' of each disk in combination
with the PCI device number. The SAS address for each drive is exposed
in sysfs as /sys/block/sdX/device/sas_address
and on our hardware
with mpt2sas appears to vary only due to the PHY
number. You can see SAS addresses in the earlier kernel messages I
gave; the first message results in a by-path filename that looks
like 'pci-0000:07:00.0-sas-0x4433221101000000-lun-0
' (for the
whole disk, and the '01
' portion appears to mark the PHY number).
Note that SAS addresses are only unique on their particular SAS controller. The system this message comes from has two SAS controllers and both controllers have disks with the SAS address of 0x4433221101000000.
(Possible interesting reading is this writeup based on what the mptsas driver does. What I've checked seems to match fairly well with what mpt2sas does on our system.)
Comments on this page:
|
|