2023-04-10
Failing to build a useful pre Go 1.21 static Go toolchain on Linux
Recently I wrote about how Go 1.21 will have a static toolchain
on Linux, where the 'go' program will
be statically linked so you can freely copy even a locally built
version from Linux distribution to Linux distribution. If you're
an innocent person, like I was before I started my journal, you might think
that achieving this yourself in Go 1.20 and earlier isn't hard. In
fact, it turns out that I failed, although my failure was disguised
by the situation in Go 1.21, where you
get a fully working static 'go' binary regardless of what you do
and whether or not it had any actual effect on the build process.
In general, there are two easy ways to get a statically linked
normal Go program, if your Go program uses only the core std
packages. First, you can build with
'(CGO_ENABLED=0))' in your environment, which completely disables
use of CGO and with it dynamically linking your Go program. Second,
you can use '-tags osusergo,netgo' to select the pure-Go versions
of the two standard packages that normally cause your Go program
to be dynamically linked by surprise.
Unfortunately, neither of these ways really works with building the
Go toolchain itself.
The easier failure is with build tags, because there's no way to pass build tags into the normal way to build Go from source. You can pass arguments to 'go tool compile', but this doesn't let you set build tags; as far as I can tell, those are controlled at a different level in the build process, in the selection of what files to compile as part of a package (see also). By the time source files are being compiled (what 'go tool compile' does), it's too late.
If you build with 'CGO_ENABLED=0' the result works in that you'll
get a statically linked Go toolchain and you can compile normal Go
programs. However, your newly built Go toolchain will never build
CGO-enabled Go executables, even if it normally would (for example if
you build a Go program using net without
setting 'netgo'). This is certainly not how you normally want a Go
toolchain to behave and it may give you real problems if you want
to build programs that require CGO to work.
The third way to build statically linked Go programs is to set the
'go tool link' flags that tell it
to create a static executable using the external linker, which are
'-extldflags=-static -linkmode=external'. What this does is
instruct Go to ask the system linker ('ext[ernal] ld') to build a
static executable. In a 'go build' command line, you pass this
with '-ldflags="..."'; when building Go itself you set this in the
'GO_LDFLAGS' environment variable. This works, but in practice
it may not do what you want, because you can't usefully statically
link a program that looks up hostnames through glibc. The 'go' toolchain needs
to look up hostnames to fetch packages, and if built without the
'netgo' tag it may try to do this through glibc, and then you need
the exact version of glibc.
(It's possible to get away with this at runtime if your nsswitch.conf is straightforward enough that Go will use its internal Go-based lookup functions, so static linking can be a step forward.)
One of the things all of this investigation has shown me is that having a statically linked Go toolchain in Go 1.21 was probably not a trivial change. That may partially explain why it wasn't done earlier.
PS: As I've found out, these days you probably have to set '-linkmode=external' in order for '-extldflags=-static' to do anything, because Go mostly seems to use its 'internal' linker. If there is a way to make Go's internal linker create static executables that are linked against glibc, I don't know what it is (and it's not the 'go tool link' -d argument). Given all of the issues with statically linking executables on Linux (and other systems), I suspect that there just isn't one.