The Unix interpreter problem(s)

December 22, 2010

A while back I mentioned that one reason I'm interested in Go is that it is compiled, not interpreted. Now it's time to elaborate on what I called 'the interpreter issue'.

First off, on Unix there are some things that interpreted languages simply cannot do (I've mentioned this before). One potentially significant limitation for an otherwise attractive high-level language is that you cannot write (script) interpreters in an interpreted language; the target of a '#! ...' in a shell script must be directly executable. Also, interpreted programs cannot be setuid.

(You can of course solve at least the first problem with another level of indirection, but.)

For me, the big issue is differences in what I'll call 'system impact' (instead of just 'performance'). A program written in an interpreted language necessarily needs to load and start the language interpreter itself before it can do anything. This costs some amount of time and some amount of memory; how much time and memory depends on the interpreter. If the program runs for a long time and uses a lot of memory itself, the interpreter's overhead is unlikely to matter. If the program is small, runs only briefly, and you want it to work fast even when the system is under significant memory pressure and load, this may matter a lot.

A compiled program is not necessarily better in this respect; the runtime environment required by an odd language (especially a high level one) may take just as much time and memory to start up as an interpreter does. But the odds are generally better that you will be able to write a fast starting program that uses only a bit of memory in a compiled language than in an interpreted one.

(In fact, these days even the standard C runtime can do quite a lot before your program gets control. But at least you have alternatives in things like dietlibc.)

Whether this matters to you depends on what sort of program you're writing, what sort of system it will run on, and what sort of situations you're going to run it in. The discussion of this deserves a separate entry; the short summary is that I don't think it's an issue any more for programs that people will run directly themselves under ordinary circumstances.


Comments on this page:

From 93.107.90.142 at 2010-12-22 04:16:12:

I did a quick test here on my Fedora 14, pentium-m 1.7GHz laptop. With a "hello world" program in perl, python, dash, bash & awk, they all show a startup time of about 2-3ms, which is the same as /bin/true on this box. Python is the exception at 70ms, but it used to be a lot worse :)

From 65.172.155.228 at 2010-12-22 17:02:18:

From personal experience I can tell you that it isn't really starting the interpreter that's the problem it's starting the "program".

For example "time python -c pass" is about 0.03 seconds, which nobody will complain about, but "yum noop" is 0.25-0.33 of a second which people do ... the difference is that yum is 32k+ lines of python, and it takes python most of the noop time to load/parse that.

If you look at IPython or sqlalchemy, or any "large" python program you see the same problem.

Written on 22 December 2010.
« Why I want tests to be easy to write
More on the Unix interpreter startup problem »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Dec 22 01:14:11 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.