Setting up self-contained Go program source that uses packages

November 27, 2020

Suppose, not entirely hypothetically, that you're writing a Go program in an environment that normally doesn't use Go. You're completely familiar with Go, with a $GOPATH and a custom Go environment and so on, so you can easily build your program. But your coworkers aren't, and you would like to give them source code that is as close to completely self-contained as possible, where they can rebuild your program with, say, 'cd /some/where; some-command' and they don't need to follow a ten-step procedure. At the same time, you'd like to use Go packages to modularize your own code so that you don't have to have everything in package main.

(You might also want to use some external packages, like golang.org/x/crypto/ssh.)

When I started thinking about this in 2018, doing this was a bit complicated. On modern versions of Go, ones with support for modules, it's gotten much simpler, at least for single programs (as opposed to a collection of them). On anything from Go 1.11 onward (I believe), what you want to do is as follows:

  • If you haven't already done so, set up a go.mod for your program and add all of the dependencies. This more or less follows Using go modules, but assumes that you already have a working program that you haven't modularized.

    go mod init cslab/ssh-validation
    go mod tidy
    

    If you don't publish your program anywhere, it's fine to give it some internal name. Otherwise you should use the official published name.

  • Vendor everything that you use:

    go mod vendor
    

  • Do modular builds using the vendored version of the packages. Not using the vendored version should work (assuming that all external packages are still there), but it will download things and clutter up your $GOPATH/pkg directory (wherever that is).

    go build -mod vendor
    

    You may want to create a Makefile that does this so that people (including you in the future) can just run 'make' instead of having to remember the extra arguments to 'go build'.

    (Since I haven't kept track of Go module support very well, I had to look up that 'go build -mod vendor' has been supported since Go 1.11, which is also the first version of Go to support modules.)

On modern versions of Go, this will automatically work right even if you have the source inside $GOPATH/src. On older versions you may need to force GO111MODULE=yes GO111MODULE=on (and so you may want to put this in your Makefile). On very old versions of Go you'll have problems, because they have either no Go module support or very limited support.

Unfortunately one of those old versions of Go is what is on Ubuntu 18.04 LTS, which ships with go 1.10.4 and has never been updated. If you're in this situation, things are much more complicated. Increasingly my view is that old versions of Go without good module support are now not very usable and you're going to need to persuade people to use updated ones. The easiest way to do this is probably to set up a tree of a suitable Go version (you can use the official binaries if you want) and then change your program's Makefile to explicitly use that local copy of Go.

PS: Use of an explicit '-mod vendor' argument may not be necessary under some circumstances; see the footnote here. I've seen somewhat inconsistent results with this, though.


Comments on this page:

If you want a more authoritative source, the Go 1.14 release notes are explicit about -mod vendor being the default if a vendor/ directory is present. It's worked exactly as advertised for me.

By cks at 2020-11-28 00:05:36:

I thought I had an instance where I needed 'go build -mod vendor', but now that I've re-tested I can't reproduce it. Sadly the default version of Go on Ubuntu 20.04 is still 1.13.8, so it'll be a while before I can safely leave off '-mod vendor' in my build instructions in our environment.

(I use a person Go 1.15.5, but my co-workers won't be if they need to rebuild a Go program we use.)

Two minor things:

  • The setting is GO111MODULE=on, yes should have no effect and leave it in auto mode
  • Technically only example/ and test/ are non-reserved by the stdlib, other names should have a dot in the first element
Written on 27 November 2020.
« The better way to make an Ubuntu 20.04 ISO that will boot on UEFI systems
The death and life of postmaster@anywhere »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Nov 27 20:04:45 2020
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.