How we handle iSCSI device names in Solaris

September 25, 2011

One of the quite important things we've done in our SAN environment is that our local software hides the real Solaris device names in favour of local labels for iSCSI devices. Hiding the real device names is not something that's normally recommended, but I feel that iSCSI is a special case, especially in a Solaris environment with MPxIO enabled.

First off, let me give you a concrete example that may illustrate why we hide device names. Here is the normal Solaris device name for a disk and the iSCSI name we use:

c2t49455400000000006F7377616C6430316935000000000000d0

oswald:disk01/0

(I think you can tell which is which.)

Yes, that's a huge device name. This is the kind of device names that you get with MPxIO, because MPxIO basically encodes a disk's identifying information in the official device name (as you might guess from the embedded hex). This is all well and good, but it means that if you use MPxIO you can easily have a whole bunch of very long names that differ in only small and basically opaque ways. Even with cut and paste it would be terribly easy to make a mistake, and I don't want to think about copying these names by hand.

The bigger problem is that Solaris device names for iSCSI devices have very low information content; with or without MPxIO, they're essentially opaque magic identifiers. To do anything much with them you need to reverse them back to determine what iSCSI target device they correspond to, because that's the only way to understand what you're doing or what's going on (or gone wrong). By contrast, our transformation of the names has very high information content; it immediately identifies the iSCSI backend, the specific physical disk on the backend, and the logical chunk from the disk.

In short, our hiding the device names transforms a mere identifier for the disk to the identity of the disk.

(This is not (and cannot be) a generic, site independent transformation. Our naming and remapping process only makes sense in our environment; in a different environment with a different iSCSI topology you would want to emphasize different things about the identity of the disk.)

There are drawbacks to doing this hiding; for a start, we need to do some extra work in scripts pretty much every time we deal with devices. Since Solaris doesn't directly export MPxIO mapping information for iSCSI devices in a machine readable format, this means we get to run iscsiadm and parse its output (which has the usual hazards). Still, I think that it's worth it; the visibility it gives us for what we're doing and what's going on is very important in practice.

(Most of our locally written tools degrade gracefully if they can't get this mapping information, displaying and accepting 'raw' Solaris device names. And of course we can always work directly with the ZFS commands, which of course only deal with Solaris device names.)

Written on 25 September 2011.
« Danger signs for mail senders in SMTP conversations
DevOps and the blame problem: an outsider's view »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Sep 25 23:44:44 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.