Why your Go programs can surprisingly be dynamically linked

November 20, 2021

Recently I read Julia Evans' Debugging a weird 'file not found' error, where the root problem was that a Linux Go program that Evans expected to be statically linked (because Go famously produces statically linked binaries) was instead dynamically linked and running in an environment without its required ELF interpreter. Although Go defaults to producing static executables when it can, winding up with a dynamically linked Go program on Linux is surprisingly common. I gave one version of the story in a tweet:

The Go standard library can need to call libc functions for a few things that it can't fully emulate, like looking up users/groups and doing hostname resolution (both can maybe require dynamically loaded NSS shared libraries for eg LDAP or mDNS). Disabling CGO turns this off.

Although it's not officially spelled out in cgo's documentation, it's well known that if you use CGO, your Go program will normally be dynamically linked against the C library. People widely assume the inverse of this; if you don't use and enable CGO (by setting CGO_ENABLED=1), you don't get CGO and so your Go program will be statically linked (well, on Linux, where Go directly makes system calls itself instead of going through the C library).

However, there are some functions in the Go standard library that intrinsically have to use the platform C library in order to work fully correctly all of the time. The largest case is anything that looks up some sort of information that goes through NSS, which can require loading and calling arbitrary C shared objects. As of Go 1.17, the two sorts of things that do are various network related lookups and user (or group) lookups. Both the os/user package and net package's section on Name Resolution mention this in their documentation, but not prominently or clearly. Each says some variant of 'when cgo is available, the cgo-based version may be used'. To simplify slightly, CGO is availble if it hasn't been specifically disabled by setting 'CGO_ENABLED=0' and you're building natively (on Linux itself.

(CGO may also be available if you're cross-compiling and have set up a relatively complex environment. Simple Go-based cross compilation doesn't normally have CGO available.)

If you don't have CGO disabled and you directly or indirectly use either net or os/user, you'll normally wind up with a dynamically linked Go executable. This executable won't necessarily actually call the C library when your program looks up hostnames (cf), but the mere possibility of needing to do it forces the dynamic linking and thus makes the program depend on the ELF interpreter for the C library you're using. Since a lot of Go programs wind up doing some sort of networking, a lot of Go programs wind up dynamically linked on Linux unless people go out of their way to avoid it.

If you want to see all the various sorts of things in the net package that can wind up making C library calls, see net/cgo_unix.go and possibly net/lookup_unix.go, which calls the stuff from cgo_unix.go under various circumstances.

PS: In the Go 1.17 toolchain (and probably in future ones), merely importing the net package will trigger this dynamic linking, even if you never call anything from it. Evidently the CGO status is a per-package thing that doesn't depend on what code you use from the package.

Written on 20 November 2021.
« What data about your NVMe drives Linux puts in sysfs
Linux has no fair-share scheduling that really works for compute servers »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Nov 20 00:46:02 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.