Wandering Thoughts archives

2023-04-10

Failing to build a useful pre Go 1.21 static Go toolchain on Linux

Recently I wrote about how Go 1.21 will have a static toolchain on Linux, where the 'go' program will be statically linked so you can freely copy even a locally built version from Linux distribution to Linux distribution. If you're an innocent person, like I was before I started my journal, you might think that achieving this yourself in Go 1.20 and earlier isn't hard. In fact, it turns out that I failed, although my failure was disguised by the situation in Go 1.21, where you get a fully working static 'go' binary regardless of what you do and whether or not it had any actual effect on the build process.

In general, there are two easy ways to get a statically linked normal Go program, if your Go program uses only the core std packages. First, you can build with '(CGO_ENABLED=0))' in your environment, which completely disables use of CGO and with it dynamically linking your Go program. Second, you can use '-tags osusergo,netgo' to select the pure-Go versions of the two standard packages that normally cause your Go program to be dynamically linked by surprise. Unfortunately, neither of these ways really works with building the Go toolchain itself.

The easier failure is with build tags, because there's no way to pass build tags into the normal way to build Go from source. You can pass arguments to 'go tool compile', but this doesn't let you set build tags; as far as I can tell, those are controlled at a different level in the build process, in the selection of what files to compile as part of a package (see also). By the time source files are being compiled (what 'go tool compile' does), it's too late.

If you build with 'CGO_ENABLED=0' the result works in that you'll get a statically linked Go toolchain and you can compile normal Go programs. However, your newly built Go toolchain will never build CGO-enabled Go executables, even if it normally would (for example if you build a Go program using net without setting 'netgo'). This is certainly not how you normally want a Go toolchain to behave and it may give you real problems if you want to build programs that require CGO to work.

The third way to build statically linked Go programs is to set the 'go tool link' flags that tell it to create a static executable using the external linker, which are '-extldflags=-static -linkmode=external'. What this does is instruct Go to ask the system linker ('ext[ernal] ld') to build a static executable. In a 'go build' command line, you pass this with '-ldflags="..."'; when building Go itself you set this in the 'GO_LDFLAGS' environment variable. This works, but in practice it may not do what you want, because you can't usefully statically link a program that looks up hostnames through glibc. The 'go' toolchain needs to look up hostnames to fetch packages, and if built without the 'netgo' tag it may try to do this through glibc, and then you need the exact version of glibc.

(It's possible to get away with this at runtime if your nsswitch.conf is straightforward enough that Go will use its internal Go-based lookup functions, so static linking can be a step forward.)

One of the things all of this investigation has shown me is that having a statically linked Go toolchain in Go 1.21 was probably not a trivial change. That may partially explain why it wasn't done earlier.

PS: As I've found out, these days you probably have to set '-linkmode=external' in order for '-extldflags=-static' to do anything, because Go mostly seems to use its 'internal' linker. If there is a way to make Go's internal linker create static executables that are linked against glibc, I don't know what it is (and it's not the 'go tool link' -d argument). Given all of the issues with statically linking executables on Linux (and other systems), I suspect that there just isn't one.

programming/GoToolchainStaticBuildFailure written at 22:22:57; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.