Building the Go compiler from source from scratch (on Unix)

May 10, 2017

Unlike some languages which are a real tedious pain to build from source, Go is both easy and interesting to build from source, even (and especially) for the latest development version. Building from source can be especially convenient if you want your own personal copy of a current version of Go (or the very latest version) on a system where you don't have permissions required to install system packages or write to /usr/local. I've seen various recipes for building Go this way, but here is the one I now recommend that you use, with some commentary on why I'm doing it this way.

First off, to build Go you need a working C compiler environment and a reasonably current version of git. Arranging for these is beyond the scope of these instructions; I'm just going to assume that you can build programs in general. Building current versions of Go also requires a working Go compiler, so the from scratch process of building Go from source needs another working Go compiler. The easiest and currently best source of this second Go compiler is a prebuilt pacakge from the Go people.

My process goes like this:

  1. Make a bootstrap area that you'll use for the bootstrap Go compiler, and fetch the latest prebuilt Go 1.8 package from the official Go downloads area:

    mkdir bootstrap
    cd bootstrap
    wget https://.../<whatever>.tar.gz
    tar -xf <whatever>.tar.gz
    

    You specifically want Go 1.8 (1.8.1 as I write this) because Go compile times took a nose dive from Go 1.5 onwards (the first version of the compiler that was written in Go instead of C) and only recently recovered. It used to be clearly slower to bootstrap Go with versions of Go from 1.5 onwards, but it's now actually slightly faster to do so with Go 1.8.1 instead of with Go 1.4, at least on 64-bit Linux x86.

    (I wound up testing this as part of writing this entry and surprised myself. I used to use Go 1.4 as the bootstrap compiler; I'm now switching to Go 1.8. A quick test suggest that Go 1.7 is also slightly faster than Go 1.4 for this, but Go 1.8 is faster than Go 1.7 so you might as well use it.)

    If your system already has a system version of Go 1.8, you can use that. If the latest version of Go is more recent than Go 1.8 (on your system or released by the Go people or both), it might be better for this. Go 1.9 is probably going to compile Go programs faster than Go 1.8, but predicting the future beyond it is hard.

  2. Get a Git clone of the current master repository:

    cd /some/where
    git clone https://go.googlesource.com/go go
    

  3. Create a little script to build your master version of Go using the version of Go in the bootstrap area; this script lives in go/src. I call my script make-all.bash, and a simple version looks like this:

    #!/bin/bash
    GOROOT_BOOTSTRAP=/some/where/bootstrap/go
    export GOROOT_BOOTSTRAP
    ./all.bash
    

    You can do this by hand but it gets to be a pain to remember the correct setting for $GOROOT_BOOTSTRAP and scripts capture knowledge.

    If you're using a system version of Go instead of your own bootstrap version, the $GOROOT_BOOTSTRAP setting you want is:

    GOROOT_BOOTSTRAP=$(/usr/bin/go env GOROOT)
    

    Or perhaps /usr/local/bin/go, or even /usr/local/go/bin/go.

  4. Build the latest version of Go with this script:

    cd go/src
    ./make-all.bash
    

    You can now add /some/where/go/bin to your path, or symlink the programs there into $HOME/bin if you prefer.

    (As with most compilers, Go does a two-stage build; first it builds itself with your bootstrap Go, and then it rebuilds itself with itself.)

When you want to (re)build the latest version of Go, you simply 'git pull' to update the master tree and then repeat step four.

Future versions of Go will make all of this somewhat easier because they'll permit you to download prebuilt binaries but put them anywhere you want without hassles. Today, it requires somewhat awkward gyrations to download one of the distribution packages but not put it in /usr/local/go, which creates more than one reason to build your own version of Go from source.

Sidebar: Building specific versions of Go

Since the development tree sometimes breaks or has things in it that you don't actually want to use, you may also want to keep around your own copy of, say, the latest officially released Go version, which is Go 1.8.x as I write this. You can do this as a Git worktree derived from your master go repository:

cd /some/where/go
git worktree add -b release-branch.go1.8 ../v1.8 origin/release-branch.go1.8
cd ../v1.8/src
cp ../go/src/make-all.bash .
./make-all.bash

('git branch -r' in your go repo will be useful here. I believe this tree can be updated when the Go people release new updates for Go 1.8, although I'm not completely sure of the best Git way to do it.)

This is different from the binary release that you downloaded to /some/where/bootstrap/go, because it doesn't require any weird steps to use. You can just add /some/where/v1.8/bin at the start of your $PATH and then everything just works, unlike the bootstrap copy, which requires you to set $GOROOT to use it.

By the way, yes, once you build your own version of Go 1.8, you can use it as the bootstrap compiler for the latest development version of Go.

(Even more recursive setups are possible. My version of Go 1.8 that I'm now using as my bootstrap Go compiler was actually bootstrapped with the latest Go development version, because why not.)

Written on 10 May 2017.
« Some things I've decided to do to improve my breaks and vacations
The challenges of recovering when unpacking archives with damage »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed May 10 03:17:33 2017
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.