In the process of writing up why we wound up using Linux for our iSCSI targets I wound up looking up the state of iSCSI target implementations on FreeBSD. It looks like they have one but it also seems to be a purely user-mode implementation (as a daemon), and when I realized that I discovered that I have an odd visceral bias against user-mode iSCSI targets and towards an in-kernel implementation.

I find this odd, partly because I don't know where this bias comes from. iSCSI is a sufficiently complex protocol to be the kind of thing I'd normally like to have out of the kernel (in code that's easier to manage and deal with), I don't have any bad experiences with a user-mode iSCSI target implementation (I don't think I've ever used one), and in fact when I looked into ATA-over-Ethernet (which you might expect me to have the same reaction to) I actually considered writing my own user-level implementation for production use.

I don't think there's any strong technical reason to (strongly) favour an in-kernel iSCSI target implementation. Depending on kernel and user level APIs such an implementation might have an easier time of doing efficient disk and network access, but I don't think it's likely to have a huge effect (in part because there's lots of spare CPU with current kernel implementations). In theory an in-kernel implementation might do less copying around of disk buffers. But given the oddities of the protocol you may well not be able to achieve this with any particular iSCSI initiator, or even in general, simply because the protocol (plus TCP) already requires you to slice and dice things around.

Despite all of this, a user-level iSCSI target implementation just makes me reflexively nervous. For no good reason I trust kernel ones more. Maybe I'm not trusting a user-level implementation to be able to manage enough concurrency to drive a big system at full capacity.

(This is one of those entries where I don't have any answers, just a realization that I want to note down. If I get the chance, I now want to explore the performance of a good user-level iSCSI target implementation.)