Wandering Thoughts archives

Some problems with iSCSI on Solaris 10 (on x86)

The basic operation of iSCSI goes something like this:

1. poke things to discover what iSCSI targets you have available, then
2. login to each target, creating sessions and discovering what iSCSI devices (eg, disks) they have available, then
3. talk to those iSCSI devices over the sessions you've created to actually do things.

There are three sorts of discovery: static configuration, SendTarget, and iSNS. Static configuration is the least convenient, because it requires you to know the target's full iSCSI name as well as its IP address; SendTarget is widely supported and just needs an IP address.

I started with Linux iSCSI, which separates things into those three phases: you use one command to discover things, another one to log in to some or all of what you've discovered (which registers the actual disks with the kernel), and then you talk with the disks themselves. (And then you can log out of a session, removing those disks from the system.)

(And yes, you can set Linux up to automatically discovery stuff and automatically log in to discovered stuff.)

Solaris 10 mashes these together, and also calls things by different names. In Solaris, discovery methods can be either enabled or disabled, which doesn't mean what you think it means; what it is actually doing is enabling or disabling logins to all targets discovered through that discovery method.

(Solaris does the actual SendTarget discovery the moment you iscsiadm add a target for ST poking; you can see what it found with iscsiadm list discovery-address -v. This is occasionally useful, for example to find out a target's iSCSI name.)

The first problem with this is logging out of sessions. On Solaris, you log out of a session by stopping discovery for the session's target. You do this either by disabling that discovery mode (which logs out of everything discovered that way) or by removing entirely the particular discovery target, which is kind of inconvenient if what you actually want to do is log out temporarily while, say, you shuffle all of an iSCSI device's logical drives around.

The next problem is that this means there is no convenient way to boot a Solaris machine and bring up sessions with just one or two iSCSI targets (for disaster recovery or whatever). Since there is no way to do a manual login to one target, you have to remove almost all of your listed targets and then enable that discovery method.

The final and biggest problem is that SendTarget discovery returns all available IP addresses for a given target, which Solaris will then try to login to all of, and Solaris 10 deals very badly with unreachable iSCSI target IP addresses, even if the target is reachable through another path. This effectively renders SendTarget discovery quite dangerous except for devices that only have a single path to them; you need to use static configuration, and I don't know how easily you can do multipath support on top of that.

(If you use SendTarget discovery, your Solaris 10 machine will not come up with the iSCSI disks accessible if you ever lose even one path to any iSCSI target. So much for redundancy. And this assumes that your machine is connected to all of the interfaces your iSCSI device has to start with, which is not necessarily true.)

The unreachable target IPs situation really irritates me, because I really thought better of Sun; it seems an obvious case to handle, yet Solaris 10 blows up relatively spectacularly. (It's not as if iSCSI devices with multiple interfaces are rare high-end gear, either.)

PS: yes, I have the current Solaris 10 on x86 recommended patch set and the latest version of the only iSCSI patch I could find.

Quality Solaris 10 software:

Very interesting things can turn up while you are testing Solaris 10 iSCSI support make sure it behaves nicely in the face of dead logical disks. Like, say:

# format
Searching for disks...
Mode sense page(3) reports nsect value as 63, adjusting it to 0
Error: can't open selected disk '/dev/rdsk/c2t1d0p0'.
Arithmetic Exception - core dumped

(I was running format to see what disks it still saw; format is often the easiest way to inventory disks on Solaris.)

Making user home directories on a stock Solaris machine

I was charmed to recently discover that Solaris 10 still hasn't fixed that useradd issue I saw with Solaris 9. For my future reference, since I am likely to wind up doing this a lot, here is how I add users on Solaris 10; this may or may not be the officially approved way to work around the useradd issue.

1. useradd -c Whatever <login>
2. mkdir /export/home/<login>
3. chown <login> /export/home/<login>

   (This may or may not get the group ownership right. It's hard to care.)

4. set the login's initial password with: passwd <login>
5. now, edit /etc/auto_home to add a line saying:

   <login> localhost:/export/home/&

   Usefully, you don't need to restart any daemons to get this to take effect.

Theoretically you can now log in to the new account and copy the default dotfiles from /etc/skel to your home directory, but as far as I can see they don't actually have anything useful so you're not actually missing anything. (Certainly they don't set a useful Solaris $PATH.)

If you want to skip all of this at the cost of some ugliness in home directory location, just manually specify that the new user's home directory is /export/home/<login> with useradd -d. I may wind up doing this for expendable VMWare installs of Solaris, since it's faster and less annoying.

(Also for my future reference, the index that man -k needs on Solaris 10 is created with catman -w.)

A gripe about sun.com

Sun's sun.com website makes a truly terrible advertisement for their hardware and software. And I'm not talking about things like their underwhelming search functionality, or how many pages you have to go through to get things; I'm talking about how slow and flaky it is. I don't want to add up how much time I've already spent trying to get a current Solaris 10 x86_64 DVD image from Sun; it's depressing.

(Also, can I say something unpleasant about offering a DVD image only in the form of a bunch of zipfiles? Honest, Sun, if I'm downloading a DVD image I can be trusted to deal with it all in one file.)

For extra points, sun.com is now insisting that I completely downloaded the file of the MD5 checksums of all of these bits and won't let me download it again, despite the fact that all I got when I clicked on their download link was a blank page in my browser.