Some problems with iSCSI on Solaris 10 (on x86)

March 27, 2007

The basic operation of iSCSI goes something like this:

  1. poke things to discover what iSCSI targets you have available, then
  2. login to each target, creating sessions and discovering what iSCSI devices (eg, disks) they have available, then
  3. talk to those iSCSI devices over the sessions you've created to actually do things.

There are three sorts of discovery: static configuration, SendTarget, and iSNS. Static configuration is the least convenient, because it requires you to know the target's full iSCSI name as well as its IP address; SendTarget is widely supported and just needs an IP address.

I started with Linux iSCSI, which separates things into those three phases: you use one command to discover things, another one to log in to some or all of what you've discovered (which registers the actual disks with the kernel), and then you talk with the disks themselves. (And then you can log out of a session, removing those disks from the system.)

(And yes, you can set Linux up to automatically discovery stuff and automatically log in to discovered stuff.)

Solaris 10 mashes these together, and also calls things by different names. In Solaris, discovery methods can be either enabled or disabled, which doesn't mean what you think it means; what it is actually doing is enabling or disabling logins to all targets discovered through that discovery method.

(Solaris does the actual SendTarget discovery the moment you iscsiadm add a target for ST poking; you can see what it found with iscsiadm list discovery-address -v. This is occasionally useful, for example to find out a target's iSCSI name.)

The first problem with this is logging out of sessions. On Solaris, you log out of a session by stopping discovery for the session's target. You do this either by disabling that discovery mode (which logs out of everything discovered that way) or by removing entirely the particular discovery target, which is kind of inconvenient if what you actually want to do is log out temporarily while, say, you shuffle all of an iSCSI device's logical drives around.

The next problem is that this means there is no convenient way to boot a Solaris machine and bring up sessions with just one or two iSCSI targets (for disaster recovery or whatever). Since there is no way to do a manual login to one target, you have to remove almost all of your listed targets and then enable that discovery method.

The final and biggest problem is that SendTarget discovery returns all available IP addresses for a given target, which Solaris will then try to login to all of, and Solaris 10 deals very badly with unreachable iSCSI target IP addresses, even if the target is reachable through another path. This effectively renders SendTarget discovery quite dangerous except for devices that only have a single path to them; you need to use static configuration, and I don't know how easily you can do multipath support on top of that.

(If you use SendTarget discovery, your Solaris 10 machine will not come up with the iSCSI disks accessible if you ever lose even one path to any iSCSI target. So much for redundancy. And this assumes that your machine is connected to all of the interfaces your iSCSI device has to start with, which is not necessarily true.)

The unreachable target IPs situation really irritates me, because I really thought better of Sun; it seems an obvious case to handle, yet Solaris 10 blows up relatively spectacularly. (It's not as if iSCSI devices with multiple interfaces are rare high-end gear, either.)

PS: yes, I have the current Solaris 10 on x86 recommended patch set and the latest version of the only iSCSI patch I could find.

Written on 27 March 2007.
« Quality Solaris 10 software:
The VPN routing problem »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Mar 27 00:00:15 2007
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.