Why realistic UDP bandwidth testing is hard

October 31, 2008

Since very few real applications just blindly fire UDP packets at a target, when you want to know what UDP bandwidth you can get you are generally interested in the speed at which you can have a UDP-based conversation between two sides. In other words, you need a bidirectional tester, not just the sort of simple unidirectional one you can use with TCP bandwidth testing.

(Of course, TCP is actually bidirectional too. It's just that the conversation inherent in TCP is already handled by your TCP stack.)

Bidirectionality is not hard in and of itself. But when you write your own code to do conversations, you have to explicitly handle all of the aspects of the conversation. This means that you have to decide things like how to behave in the face of delayed or dropped packets (how do you notice? what is your retry policy?) and how much you're going to do in parallel and how (are you going to have an outstanding window size or multiple simultaneous outstanding requests? the two are subtly different).

And this is the real problem: you are presumably testing your UDP bandwidth because you are trying to assess how some real application or protocol will perform. Because everyone has to answer these questions themselves when writing UDP-based protocols, different applications can come up with quite different ones. If you do not match how your tester deals with these issues with how your target application does, your performance numbers may not actually match the real world, which means that your tester is not much good.

So: how do you find out what your target protocol does (or will do) in these situations, so that you can faithfully imitate it in your tester (in all of its possible complexity)?

All I can say is 'good luck with that', because now you know why realistic UDP bandwidth testing is hard. It's not the coding; it's figuring out what to code in order to get meaningful answers.

(Speaking from personal experience, it is very easy to create a UDP bandwidth tester that gives hopelessly optimistic and meaningless answers. And it's probably equally easy to create one that gives hopelessly pessimistic answers.)

Written on 31 October 2008.
« Banging rocks together in Python
Why university webmail systems are attractive to spammers »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Oct 31 23:08:23 2008
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.