Wandering Thoughts archives

Understanding the Solaris iSCSI initiator (a bit)

If you're an innocent person (like I used to be), the Solaris iSCSI initiator appears much like how it works on other Unixes. You have an administrative command (iscsiadm), a system daemon (iscsid, which is what the SMF service svc:/network/iscsi/initiator starts), and a kernel component that presumably turns iSCSI connections into SCSI disks. Unfortunately this view of Solaris is highly misleading, or as I should actually admit, wrong.

Because of the complexity involved, most systems split the iSCSI initiator into a user-level system daemon that does target discovery, iSCSI login, and session initiation and a kernel component that takes established iSCSI sessions and does iSCSI IO with them. Solaris does not work this way.

In Solaris, the entire iSCSI protocol stack is in the kernel, including all target discovery. Yes, this includes the extra protocols used for finding targets (iSNS and SendTargets). That tempting looking iscsid daemon actually only has two little jobs: it tells the kernel to start up the iSCSI initiator (and keep it running) and it does hostname lookups for the kernel. Oh, and it tries to avoid reporting 'service ready' to SMF until the kernel seems to have completed iSCSI discovery or discovery has stalled out.

(iscsid does not even read and write the iSCSI initiator configuration database in /etc/iscsi; the kernel does it directly. By the way, the database is stored as a serialized nvlist (of course). Normally there are two copies, the current database and the previous database.)

None of this is documented, of course, or at best it's only documented if you read carefully between the lines in the way that the Solaris people want you to.

PS: according to comments in the OpenSolaris iscsid code, the hostname lookup is incomplete. iscsid only returns to the kernel a single IP address for a hostname, regardless of how many the host has; it picks the first one that the underlying library call returns.

Sidebar: when iscsid reports things to SMF

Because I was just looking at this in the source code and we may need it sometime: first, if the kernel reports that all forms of iSCSI target discovery have completed, service startup is obviously done. After that iscsid gives up and declares 'service started' if it's been 60 seconds without any new LUNs being discovered. As long as you discover at least one LUN every minute, SMF will keep waiting for svc:/network/iscsi/initiator to complete.

(What effect this has on the rest of the system is unclear, since nothing depends on the iSCSI initiator in SMF from what I can see.)