If you write a simple threaded program in C on a modern Linux system and fire it up, you may rapidly notice that you can't have more than a couple hundred threads before you run out of memory. Even if you don't need that many threads, you may discover that your program is using an absurd amount of (virtual) memory.

(This can also happen in other languages; I know it happens in Python.)

What's eating all the memory is the default NPTL thread stack size. If your program doesn't specify a stack size for its new threads, NPTL defaults to your 'stack size' resource limit (what 'ulimit -s' reports in bash, aka RLIMIT_STACK); if your stack size is set to unlimited, NPTL picks two megabytes. NPTL only looks at your RLIMIT_STACK once, very early during program startup (well before the C main() gets control).

(NPTL is the 'native Posix thread library' for Linux, designed to use the new kernel support routines that appeared in the 2.6 kernel series. If you're using a distribution based on the 2.6 kernel (or Red Hat 8 or later, including RHEL 3 I believe), you're almost certainly using NPTL for threads.)

There's two obvious ways out:

1. set a non-default stack size via pthread_attr_init and pthread_attr_setstack, if you have access to that level of control over pthread_create's usage.
2. crank your stack size resource limit down before starting the program; this works on anything.

If you spawn other programs, beware of setting your stack size too low; 128K is about the minimum I would go these days. (I definitely saw core dumps at 64K.)

The gory hack variant of #2 is to crank the stack size resource limit down before starting your program and then have your program crank it right back up to where it should be once it gets control. This is useful for environments (such as Python) where you don't have control over pthread_create parameters but you can fiddle with the resource limits.

(In Python, you want the resource module.)