A common socket programming mistake: not handling short IO

April 8, 2006

With normal file IO, when you do do 'read(fd, buf, len)' you'll almost always get back len bytes unless you hit EOF or a disk IO error. This breeds a certain sloppyness when filling buffers; an awful lot of code effectively ignores the return value of read() except to check it for errors.

This can and will bite you on the rear when writing socket code, because networks only give you so much data at once. Short reads are routine for socket IO; you can't assume that you can get all of what you want in a single read.

The mistake is especially pernicious because the mistaken code almost always works. Usually the lines or transactions you're reading from the network are small; usually you test on a fast local network. Speaking from personal experience, it's easy to forget this and then not notice.

(Today's case was some of my code that assumed it could read all of a HTTP POST body in one read().)

Whether or not a normal (blocking) write() has similar issues is probably system dependent. Linux seems to only return from a socket write() once all data has been pushed out, but I don't know what other systems do in practice.

(According to the Single Unix Specification page for write(), in theory you can count on this behavior on any SuS compliant system.)

Written on 08 April 2006.
« A bash irritation
Apple joins the webmail hall of shame »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Apr 8 00:51:49 2006
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.