'Argument list too long' is a misleading message
If you try to run a command with too many command line arguments, you
get an 'Argument list too long' error (technically you get an
error from the kernel, which is then mapped into this error string by
your shell). There's a bunch of workarounds for this problem, such as
However, the error message is actually somewhat misleading. As old
Unix hands know (and new ones generally don't), the kernel's size
exec() is on the combination of arguments and environment
variables. If you accidentally wind up with a huge environment and try
to start a program with even a few arguments, you'll fall over. (And if the
environment is big enough, you can't start any programs at all.)
Modern shells make this a more mysterious failure since they have so many built in commands, which means that you can get a substantial way into a shell script before you try to run an external command and fail. The net result is that you can spend a bunch of time scratching your head and trying to figure out why an innocent command with only a couple of arguments is getting this error.
(If you need to figure out what the big environment variables are in the
Bourne shell, it is useful to know that the
export command without
any arguments lists all the environment variables, and it is a built-in
command so you can still use it in this situation. If you are trying to
tcsh, you're on your own.)
Sidebar: why I'm so aware of this issue
Most people run into this issue only very rarely, because you need both a very large shell variable or three and to have them exported to the environment. Just pulling an unexpectedly large amount of data into a variable to process it won't hit this. However, I use a shell that automatically exports all shell variables into the environment, so any time I pull a big value into a shell variable I run into this.
My workaround is simple; when I run external commands I null the variable for the command, like so:
BIGVAR='' command ...
This is normally used to add some environment variables just for a command, but it works fine to take them away too.
(This also works in the Bourne shell, although you are much less likely to need it.)
Comments on this page:Written on 12 June 2007.