What you can and can't build in Go's module mode
Go modules are soon going to be our only option for building Go
programs, which means that it's useful to
understand what we can and can't build with Go in 'module mode',
and how to do certain customary things as part of this. There are
a lot of articles on using Go modules as a developer, but my major
use of '
go get' and friends is to build and consume other people's
programs, so that's what I'm focusing on here.
(Today in Go 1.15, you're in module mode if you use '
or are inside a directory tree with a
go.mod. When Go 1.16 is
released soon, you will be in module mode all of the time by default,
and in Go 1.17 you won't even have the option to be in GOPATH mode,
as covered before.)
To just install a binary for a Go program in $HOME/go/bin directly
from the upstream source, you do '
go get <thing>' (provided that
you're not in any source repository for a Go module). This works
both for programs that are Go modules (or part of ones) and non-modular
programs, even if the non-modular Go programs use third party
dependencies. If there are any version tags in the source repository,
you get what Go considers to be the most recent version (I believe
including v2.x and so on versions). Otherwise, you get the most
recent commit on the main branch. I don't believe there's any way
to use this form of '
go get' to deliberately build an in-progress
development version instead of a tagged release once the latter
exists. If a new version is released, I believe that re-running
go get <thing>' will update you to the new latest version.
(There is a '
go get <thing>@latest' syntax, but it doesn't appear to
do anything different than plain '
go get <thing>' in this case.)
You can also use '
go install <thing>@latest' to do this in current
(and future) Go versions, which has the advantage today that it
always works in module mode (or fails outright). In the future, the
Go developers plan to remove '
go get's support for actually getting
and building Go programs, leaving '
go install <thing>@latest' as
the only option. This means people can look forward to years of
annoyance from trying to follow the
READMEs on old and perfectly
useful Go programs (including some modularized ones).
If you have a local source repository of a Go program that's part
of a Go module, you can build the current version in the repo by
cd /where/ever; go install -mod readonly'. It's important to
-mod readonly' unless you're actually developing the package,
because otherwise Go's tooling can make changes that will cause
conflicts in future VCS updates. The local source repository doesn't
have to be under $HOME/go/src.
If you have a local source repository of a Go program that hasn't
been modularized, it's likely that you can't build from this source
repository in Go 1.16 without special steps and you won't be able
to build from it in Go 1.17 at all (if the Go developers stick to
their plan). In Go 1.16, because GOPATH mode remains supported, you
can do non-modular builds in $HOME/go/src with '
go get <thing>' (or '
GO111MODULE=off go install' in the right
directory). If you think that the upstream will not update the
program to make it a Go module, you can do '
go mod init <thing>'
to create your own local
go.mod and modularize the program.
Modularizing the program yourself will be the only option once
GOPATH mode is no longer supported at all.
(This means that it will be easier to build old un-modularized
Go programs directly from Github et al than from a local copy, since
go install ...@latest' will still work in the former case. I
sure hope those upstream repositories never get removed.)
In module mode, there's no way to use '
go get' to clone the source
repository of a program, whether or not it's modular. While Go still
supports non-modular mode, you can force Go to clone repos into
$HOME/go/src with '
GO111MODULE=off go get -d <thing>'. As far as
I know there's no standard Go tool that will tell you the actual
source repository and VCS used for a given Go package path, so that
you can readily deal with custom import paths (a 'vanity import
there will be someday. Similarly, if you want to fetch the latest
updates you must directly use an appropriate VCS command from within
the source repository. This is usually '
git pull --ff-only', but
there are still some people using Mercurial and other alternatives
so it's up to you to keep track of it.
(If you have the source repo in the right spot under $HOME/go/src, in
Go 1.16 you can force an update with '
GO111MODULE=off go get -u -d
<thing>'. This will also update your local copy of any dependencies,
but you probably don't care about that.)
If you were previously tracking the development of an upstream
program by doing '
go get -u <thing>' periodically, you now need
to do a multi step process to update and build the latest development
cd /where/ever || exit 0 git pull --ff-only # (probably) go install -mod readonly
You'll also need to manually clone the repository the first time around. Although Go downloads the source code for modules, it's just the source code for whatever version you're using, not the full source repository (and Go doesn't normally expose it to you anyway).
(If you put your cloned source repositories in the right place under
$HOME/go/src, you can use
gostatus to check if they're out of
date. Otherwise, there's '
git fetch --dry-run', although that's
pretty verbose if there are updates. Perhaps someone will write or
has already written a Git remote status checking program like
gostatus that works on arbitrary directories.)
If you just want to periodically update to the latest released
version of a program, if any, and perhaps rebuild the version with
your current version of Go, I believe that
go install <thing>@latest' will always do what you want. Further,
given the module information that's embedded in binaries compiled
in module mode, you can recover the necessary '<thing>' from the
A Go binary that was built in module mode carries module information
in it that can be reported by '
go version -m', which will give
you the module and package of the main program. This includes
non-modularized programs fetched and built directly in module mode
go install <thing>@latest' (or '
go get <thing>' while
that still works). However, the reported information does not include
the local path of the source code. If you need to get such paths
once in a while, probably the simplest way today is to use Delve:
$ dlv exec someprogram Type 'help' for list of commands. (dlv) ls main.main Showing <path>/<something>.go: [....]
(As I found out when I looked into it, there's a lot of complexity in determining this information from a Go binary.)