Possible limits on our port multiplied ESATA performance
June 27, 2009
Our iSCSI targets use port multiplied ESATA to talk to their data disks. This, for those who have not had to deal with it yet, is a way of undoing one of the advantages of SATA; instead of having one disk per port, channel and cable, now you have some number of disks per port (I've seen as many as five). The advantage is that if you want lots of disks, you don't have to find room for (say) 12 individual connectors on the back of your enclosure, and to somehow get 12 individual ESATA ports into your system (with the resulting proliferation of PCI cards); instead you can just have three or four.
It's clear that we are running into some sort of overall bandwidth limit when doing streaming reads. The question is where. I can think of a number of possible limits:
Fortunately, this performance shortfall is basically irrelevant in our environment; for sequential IO, the iSCSI targets will be limited by total network bandwidth well before they'll run into this limit, and for random IO you care far more about IO operations a second (per disk) than you do about bandwidth.
(We are not rich enough to afford 10G Ethernet. And I have seen an iSCSI target easily do 200 Mbytes/sec of read IO, saturating both of its iSCSI networks.)
Looking into this has shown me that I don't know as much about the current state of PC hardware and its performance characteristics as I'd like to. (Yes, I know, it's sort of like the weather; wait a bit and it will change again.)
* * *
Atom feeds are available; see the bottom of most pages.