A thought about static linking and popularity

January 11, 2013

This all started with my entry that touched on packaging with Go. Go is attractive partly because it what you get at the end is a basically self-contained program with no dependencies (unlike in, say, interpreted languages). One part of this is that Go static-links everything.

The major advantage of static linking is no runtime dependencies. The disadvantages of static linking are that updates to packages (in the Go sense, ie a component or a library) are more of a pain, that the accumulated collection of everything using a package takes up more disk space, and that that a bunch of things running at the same time that use the same package can't share its memory (each gets its own copy).

As I was thinking about this, it occurred to me that the more popular your static-linked environment is the more that the drawbacks of static linking are a problem. If you only have a few programs written in Go, it's not a particularly big issue to rebuild and reinstall things if (or when) a bug or security issue comes up, and you wouldn't save much disk space or memory anyways. But imagine if Go becomes a popular language for writing programs in, to the point where a typical Linux system has a couple of hundred static-linked Go programs; all of these drawbacks would be much larger. A bugfix to a core package would result in a huge update, for example, because many programs use it and would have to be rebuilt.

(I suspect that the authors of Go don't really care about it being popular in this way and don't expect anything like it to happen any time soon, so I doubt that this issue is something that even crosses their radar.)

Another way to put this is that most of the time, static versus dynamic linking is a system engineering issue instead of a language or a programming one. The system administrators and system builders want it, while programmers don't care (except as a neat technical challenge).

(There are uses for being able to dynamically load code, even just at startup, but many programs don't need anything like that. Even when they do, these days things like embeddable JIT'd 'interpreted' languages may offer alternate solutions.)

Comments on this page:

From at 2013-01-11 03:20:03:

One of the more irritating aspects of static linking in a C/C++ world is that you have to worry about transitive dependencies because a static library has no way of storing a list of what it depends on. You also have to be careful about the ordering of libraries on the link line. Are these issues somehow solved in Go?

It's interesting to think that dynamic linking may go out of fashion perhaps as more memory means that we don't need it so much. It's certainly more complicated. I've lost count of how many times I've had to explain library version numbers to colleagues. Even if you understand the issues it can be far from trivial to make updates without breaking existing software. The way symbols are bound in a static program is much simpler.

From at 2013-01-11 03:33:17:

Google statically links everything. Go is developed at Google. Go statically links everything. Coincidence?

From at 2013-01-11 10:38:36:

As I was thinking about this, it occurred to me that the more popular your static-linked environment is the more that the drawbacks of static linking are a problem.

That's a nice way to restate the reasons static libc was abandoned. libc is very "popular" - everything links against it.

Jeff Sipek.

By cks at 2013-01-11 10:50:51:

It turns out that Go was deliberately designed and engineered to solve some of these dependency issues, among other software engineering ones. There's a quite interesting Rob Pike talk and article about this that's we ll worth reading, Go at Google: Language Design in the Service of Software Engineering. Reading between the lines of his article, it seems clear that the Go equivalent of static libraries almost certainly includes information about everything needed to link the static library (including its dependencies).

(In Go, as in many other modern languages, you have to explicitly import packages in order to use things from them. This means that the compiler can completely discover dependencies; it's just a matter of recording them. C's dependency hell for static linking comes from its lack of any such import step; there's only a single global namespace and names in it can magically materialize from anywhere.)

From at 2013-01-11 10:59:03:

As an application developer, the idea of static linking comforts me because it raises the chances that if someone is going to patch one of my dependencies, they're going to go through a complete compile/test cycle of my application. This should catch any inadvertent incompatibilities introduced by the patch.

(That said, I'm not too familiar with how linking works in C. I'm a java developer, where basically all linking is dynamic. Sometimes people drop in incompatible jar files and then ask me what the scary stacktraces in the logs are all about.)

From at 2013-01-11 17:34:52:

As a packager (mostly ruby stuff), I find the idea of statically linking everything very interesting. It would make my life much easier (ever tried to properly package chef on RHEL systems? I do...).

One thing I'm not sure though (as I haven't looked into it) is wether Go programs supports some kind of plugin system, say for example if you wanted to develop a daemon that would connect MySQL and/or PostgreSQL and not link to both client libraries.

By cks at 2013-01-14 11:42:36:

I don't think that Go currently supports any sort of native dynamically loaded plugin system. At most you might be able to directly load and call non-Go dynamic libraries (although you'd probably need some sort of FFI support) via interfaces to dlopen() and dlsym() and so on.

Written on 11 January 2013.
« The fundamental problem faced by user-level NFS servers
Runtime loading of general code usually requires general dynamic linking »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Jan 11 00:20:40 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.