A little neat detail of the BitTorrent protocol

October 24, 2008

To simplify, BitTorrent breaks files up into blocks, checksums each block, and then has all of the clients (the peers) swap blocks back and forth. The checksums for all the blocks are in the .torrent metainfo file that each client has, so you know when you get a bad copy of a block (and as a side effect, a malicious client can't serve you trojan data).

(One might think that trojan data would be relatively harmless. But BitTorrent is used to distribute things that are then executed, such as bootable ISO images of Linux distributions; allowing an attacker to substitute their own version of such things would be very bad.)

Well, sort of. One of the little neat bits of the BitTorrent protocol is that there are actually two different blocks and two different block sizes: the metainfo block size and the peer transfer block size. The metainfo block size is set when you create the torrent metainfo file and is usually fairly large; the peer transfer block size is decided by the peers, and is usually much smaller than the metainfo block size.

(The typical peer block size is 16 KB; the typical metainfo block size ranges from 256 KB to 1 MB. I'm using 'block' for both here, but the official term for metainfo blocks is 'pieces'.)

The block size difference is ultimately because the goals of the two sorts of block sizes conflict. You want a big metainfo block size so that you don't have very many checksums and you keep the size of torrent metainfo files down, while you want a small peer block size so that peers do small quick transfers instead of big long ones. The small peer block size has other consequences, including that you can get the parts of a single metainfo block from several peers at once, which is important when peers often have asymmetric bandwidth with slow upstream rates.

Written on 24 October 2008.
« Another update to the ZFS excessive prefetching situation
How we worked out the partition sizes for our SAN »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Oct 24 00:55:56 2008
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.