Wandering Thoughts archives


Controlling Linux TCP socket send buffer sizes

Suppose that you are dealing with some piece of code that uses TCP sockets and does not set an explicit send buffer size. You vaguely remember that the default send buffer size can be tuned, but nothing immediately obvious turns up. This is the situation I found myself in recently.

It turns out there are two answers. The first one is /proc/sys/net/ipv4/tcp_wmem, which is mostly written up in Documentation/networking/ip-sysctl.txt in the kernel source. You want to change the middle number (which is in bytes). Note that this is not documented where you might expect to find it, in Documentation/sysctl/net.txt (although Documentation/sysctl/README sort of has a pointer to ip-sysctl.txt).

The second answer, the one I found out the hard way, is that you probably don't want to change this even if you think you do. You see, the low default number doesn't matter in practice because the Linux kernel auto-tunes the TCP send buffer size on the fly (unless your code explicitly sets the send buffer size). Assuming that your network is working right, a TCP socket's send buffer size will normally open right up to the amount of data that the connection can handle and your code should never notice any cases of insufficient send buffer size.

The corollary is that unless you are doing something unusual with your networking, noticing an insufficient send buffer size is actually a sign of underlying network problems. The reason your send buffer is too small is that something is going on the network that makes the kernel think it can't tune the buffer size up. You can increase the send buffer size anyways by increasing the default (even increasing it drastically), but all this will do is push the problem down a layer where you can't see it as easily any more. You'll still have a network problem. What you really want to do is find the network problem.

(If you think that you don't have a network problem, you should be able to come up with a convincing explanation of why and how normal TCP send buffer size scaling doesn't work in your situation. Of course, as always experimental results trump my meanderings so feel free to try it even if you don't have such an explanation; just don't be surprised if nothing improves.)

linux/TCPSendbufferDefaultSize written at 00:28:31; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.