As I've written before, I don't think we're going to see any great use of 10G Ethernet in our immediate future; it's still too expensive to see it routinely start showing up on machines. But I've recently been thinking about what we could feasibly use 10G Ethernet for in the relatively near future, if (for example) we got money to buy a modest amount of 10G gear.

(I think that this is an interesting question in general. For the moderately near future, many places will be able to afford a few 10G connections but only a few will be able to afford a significant number of them. So the question is: what can you use a modest number of 10G connections for that makes sense?)

The nominally obvious place to deploy 10G Ethernet is in a fileserver infrastructure. Ignoring any lack of need for it, the problem is that when you have a SAN, increasing the speed of one network connection doesn't really help because you just wind up bandwidth constrained on other connections. We could give our fileservers 10G connections to the outside world but they'd only be able to use a fraction of it, since they only have two 1G connections into the SAN fabric. If we gave them 10G SAN links, the backends need fast connections or we max out at four 1G connections (at best). All of this adds up to a lot of 10G connections (and a lot of changes to our systems, but let's ignore that).

A lot of the other places that we could sprinkle 10G connections over have similar issues; isolated 10G connections between a few machines generally don't do us any good, because we don't have any super-hot, bandwidth constrained machines. This is in a sense the downside of building a balanced architecture; lifting it up requires improving a bunch of places at once.

However, there is one place that we could usefully deploy a small number of 10G ports. The core of our physical network is a set of switches chained together through pairs of 10G ports (each switch has two). This is an increasingly awkward architecture as time goes by (for reasons beyond the scope of this entry). We would love to move to a star model where there is a central 10G switch that all of the core switches uplink to; at this point even a four-port 10G switch would give us a useful simplification of the topology.

(Given an increasing use of 10G for switch to switch links, I suspect that this will be a major way that 10G works its way into most machine rooms. We can't be alone in having this topology problem.)