Why I think 10G-T will be the dominant form of 10G Ethernet

March 8, 2014

Today there are basically two options for 10G Ethernet products and interfaces, 10G-T (standard Ethernet ports and relatively normal Ethernet cables) and SFP+ (pluggable modules mostly using fiber). Historically SFP+-based products have been the dominant ones and some places have very large deployments of them while 10G-T seems to only have started becoming readily available recently. Despite this I believe that 10G-T is going to be the winning 10G format. There are two major 10G-T advantages that I think are going to drive this.

The first advantage is that 10G-T ports are simpler, smaller, and cheaper (at least potentially). SFP+ ports intrinsically require additional physical modules with their own circuitry plus a mechanical and electronic assembly to plug them into. This adds cost and it also adds physical space (especially depth) over what an Ethernet RJ45 connector and its circuitry require. In addition 10G-T is pretty much just an RJ45 connector and a chipset, and the hardware world is very good at driving down the price of chipsets over time. SFP+s do not have this simplicity and as such I don't think they can tap quite this price reduction power.

The second advantage is that 10G-T ports are backwards compatible with slower Ethernet while SFP+ ports talk only with other SFP+ ports. The really important aspect of this is that it's safe for manufacturers to replace 1G Ethernet ports with 10G-T Ethernet ports on servers (and on switches, for that matter). You can then buy such a 10G-T equipped server and drop it into your existing 1G infrastructure without any hassle. The same is not true if the manufacturer replaced 1G ports with SFP+ ports; suddenly you would need SFP+ modules (and cables) and a bunch of SFP+ switch ports that you probably don't have right now.

In short going from 1G to 10G-T is no big deal while going from 1G to SFP+ is a big, serious commitment where a bunch of things change.

This matters because server makers and their customers (ie, us) like 'no big deal' shifts but are very reluctant to make big serious commitments. That 10G-T is no big deal means that server makers can shift to offering it and people can shift to buying it. This drives a virtuous circle where more volume drives down the cost of 10G-T chipsets and hardware, which puts them in more places, which drives adoption of 10G-T as more and more equipment is 10G-T capable and so on and so forth. This is exactly the shift that I think will drive 10G-T to dominance.

I don't expect 10G-T to become dominant by replacing existing or future enterprise SFP+ deployments. I expect 10G-T to become dominant by replacing everyone's existing 1G deployments and eventually becoming as common as 1G is today. Enterprises are big, but the real volume is outside of them.

By the way: this is not a theoretical pattern. This is exactly the adoption shift that I got to watch with 1G Ethernet. Servers started shipping with some or all 1G ports instead of 100M ports, this drove demand for 1G switch ports, then switches started getting more and more 1G ports, and eventually we reached the point we're at today where random cheap hardware probably has a 1G port because why not; volume has driven the extra chipset cost to basically nothing.

Update: The reddit discussion of this entry has a bunch of interesting stuff about various aspects of this and 10G Ethernet in general. I found it usefully educational.


Comments on this page:

By Pete at 2014-03-08 21:52:00:

What's more, 1G had its own SFP+, called GBIC, and I saw deployments of those. They shared the cable plant and plug-in modules with Fibre Channel.

By James (trs80) at 2014-03-08 22:21:58:

The difference this time is you can't get a 10G-T SFP+ module (as 10G-T draws more power than SFP+ provides), whereas you could get 1000Base-T GBICs. This makes the upgrade path less seamless for existing SFP+ customers.

For example, I have some Dell servers bought last year (or maybe even late 2012?) that could have come with 10G-T onboard, but I went with SFP+ instead (with Twinax SFP+ copper cabling), because we only had SFP+ ports in our existing switches.

Once they come out with 10G-T chipsets that fit in the SFP+ power envelope, I agree with you that 10G-T will win, but until then it'll be slower than the 100Mbps -> 1Gbps transition.

By Ben Hutchings at 2014-03-09 08:33:47:

I worked with both for many years at Solarflare, which had some of the first 10GBASE-T PHYs and boards on the market starting in 2007. SFP+ came along a year or so later, as I recall, or maybe we were a little late to support it. Anyway, I was expecting things to progress as you say, but it never seemed to happen. In the end, Solarflare more or less got out of 10GBASE-T, although it still has some boards available.

The key problem is that 10GBASE-T requires complex signal processing to work reliably in the worst case (100 metre cable with 4 connectors and 6 more cables right next to it), and that takes both a lot more silicon and good deal more power than Direct Attach or 10GBASE-SR. When we combined a 10G controller and PHY into a single chip, the (single-port) PHY used more power and die area than the (dual-port) controller.

Power usage and die area should come down over time (and I know power usage came down a lot while I was following development of the PHYs) but there will always be a premium over SFP+ (though for the end user the cost of cables outweighs this).

But we're talking about 10G being a ubiquitous commodity item, and it's going to have to be really cheap before that happens. The host bandwidth requirements (PCIe gen3 2x or gen2 4x) are also still a problem for a desktop/laptop system, as they tend not to have much to spare after allowing for a GPU - see http://en.wikipedia.org/wiki/LGA_1150 for the current Intel desktop specs. And I don't know what the killer app will be that actually wants much more than 1G to the desktop. Is HD video editing common enough yet?

I can't believe 10G will never reach the desktop, but I gave up waiting.

- Ben Hutchings

By James (trs80) at 2014-03-10 09:38:27:

Something slightly more likely than 10G-T everywhere is 2G-T or 4G-T or similar. The driver for this is the theoretical bandwidth of phase two 802.11ac access points, which is more than a 1Gbps, but much less than 10Gbps.

Written on 08 March 2014.
« Coming to terms with D-Bus
Why we don't change Unix login names for people »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Mar 8 02:04:39 2014
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.