2014-02-19
Some rough things about the naming of SAS drives on Linux
This isn't a comprehensive look at the names of SAS drives because SAS (as a technology) has a lot of options. Instead this is a look at how directly attached SATA drives behind SAS controllers get named. I assume that real SAS drives get named the same (but I don't have any to test with) and I have no idea how things look for SAS (or SATA) drives behind a SAS expander or more complex SAS topologies. Also some of this depends on the SAS controller and its driver; we're using LSI cards driven by the mpt2sas driver.
To start with, SAS drives have sdX names and kernel 'SCSI host names'
just like SATA drives do. As with SATA drives, sdX drive names are
not necessarily stable across hotswaps. Unlike SATA drives, the SCSI
host names are not necessarily fully stable either; I've seen them
get renumbered within a SCSI host after a hotswap, so that what was
'scsi6:0:3:0' becomes 'scsi6:0:8:0'. So far, LSI cards have a single
SCSI host regardless of how many SAS ports they support and how those
ports are physically connected (eg with a 4x SFF-8087 multiplexer or
broken out as individual SAS ports).
(For background, see my basic overview of SAS and the names of Linux SATA devices, which covers kernel SCSI host names in more detail along with the three other device namespaces.)
All of this tells you nothing about the drive's physical slot. In SAS (or at least in mpt2sas for directly attached devices), physical location information is the SAS PHY number. To find this you must go grubbing around in sysfs, so I will just show you:
set -P # in bash cd /sys/block/sdX/device cd ../../.. ls phy-*
(There is probably a simpler path from the block device, but sysfs is the kind of thing where I find a working way and then stop.)
The PHY name will be of the form 'phy-6:0', meaning PHY 0 on SCSI host
6. PHY numbers sometimes also show up in kernel messages, such as:
scsi 6:0:2:0: SATA: handle(0x000a), sas_addr(0x4433221101000000), phy(1), device_name(0x50014ee25df6de01)port-6:7: remove: sas_addr(0x4433221103000000), phy(3)
Mapping PHY numbers to actual physical hardware slots is something that you'll have to do yourself for your specific hardware. Please don't assume that PHY numbering matches how the card BIOS does it (as seen in BIOS information printouts during boot or if you go into the BIOS itself); for our LSI cards, it does not.
(Although it may be obvious, PHY numbers are reused between different SAS controllers. Several controllers may all have a PHY 0.)
Since the SCSI host name of a SAS drive in a given physical slot
is not stable, /dev/disk/by-path sensibly does not use it for SAS
drives. Instead it uses the 'SAS address' of each disk in combination
with the PCI device number. The SAS address for each drive is exposed
in sysfs as /sys/block/sdX/device/sas_address and on our hardware
with mpt2sas appears to vary only due to the PHY
number. You can see SAS addresses in the earlier kernel messages I
gave; the first message results in a by-path filename that looks
like 'pci-0000:07:00.0-sas-0x4433221101000000-lun-0' (for the
whole disk, and the '01' portion appears to mark the PHY number).
Note that SAS addresses are only unique on their particular SAS controller. The system this message comes from has two SAS controllers and both controllers have disks with the SAS address of 0x4433221101000000.
(Possible interesting reading is this writeup based on what the mptsas driver does. What I've checked seems to match fairly well with what mpt2sas does on our system.)
The reasoning problem with describing things with a programming language
Every so often systems need to have things described to them. Configuration files are one obvious example but there are plenty of others, such as what to do to build software, what to do when packages are installed or removed, and what to do when you're starting and stopping services. When you're creating such a system it's often tempting to make the description take the form of a program written in some programming language, partly because this is often the easy way to create domain specific languages and partly because often it's easy to think of what you are doing as less describing things and more making things happen.
Unfortunately there is a problem with this. The more you describe things through a programming language (and the more general the programming language is), the less your system can determine things about the description from the outside. Actually, let's call the description what it is: a script. Often you can't determine very much about what the script does without in some way executing the script itself. If you're unlucky, you can't determine much without the script actually running for real and doing whatever it does. And even if you have a good 'dry run' mode you generally can't capture why the script did what it did except at a very low level of 'the script made this series of conditional decisions before it acted'.
What this leaves you with is a relatively black box and the problem with black boxes is that there's not much you can do to reason about them. For example, let's take package management. Suppose you have a bunch of similar packages to install or update and they have a bunch of postinstall scripts between them. Are there common operations across these postinstall scripts that you can perform only once, such as an expensive regeneration of a system-wide MIME database? You don't know. You don't even necessarily know this if the postinstall scripts are all identical (depending in part on package manager semantics).
While there are cases where you need the full arbitrary power of a programming language, you should try very hard to avoid resorting to a programming language for everything. White-box as much of your descriptions as possible and reserve the general programming language as a last ditch escape hatch. If people are using the programming language very much it's a sign that your descriptions lack enough power and you need to do something about that.
(There is also a usability problem with using programming languages for configurations and descriptions.)
Sidebar: gradiations of describing things
The best way of describing things is declaratively: you directly declare X, Y, and Z.
The second best way is a procedural language that winds up making declarative statements. You run a chunk of code and it ends up declaring X, Y, and Z. Configuration files written in programming languages often wind up doing this (where the 'declaring' may be in the form of, say, setting specific variables). Among other issues, this suffers from the problem that you know the end declarations but you don't know very much about why they wound up that way.
The worst way is a procedural language that takes actions; the code just does X, Y, and Z. Here you can only discover what is being described by running the code and watching what it does (if you even bother watching, as opposed to just standing back and letting it act).