A sysadmin's perspective on Go vendoring and vgo

March 2, 2018

One big thing in the Go world lately has been Russ Cox's writings on adding package versioning to the core of Go through what is currently being called Versioned Go, vgo for short. His initial plans were for vgo to completely drop Go's current vendoring feature. If you wanted to capture a local copy of your external dependencies, you would have to set up your own proxy server (per his article on modules, vgo would come with one). According to the vgo & vendoring golang-dev thread (via), opinions have since changed on this and the Go team accepts that some form of vendoring will stay. My interest in vendoring is probably different from what normal Go developers care about, so I want to explain my usage case, why vendoring is important to us, and why the initial proxy solution would not have made me very happy.

We are never going to be doing ongoing Go development, with a nice collection of Go programs and tooling that we work on regularly and build frequently. Instead, we're going to have a few programs written in Go because Go is the right language (enough so to overcome our usual policy against it). If we're going to have local software in a compiled language, we need to be able to rebuild it on demand, just in case (otherwise it's a ticking time bomb). More specifically, we want people who aren't Go specialists to be able to reliably rebuild the program following some simple and robust process. The closer the process is to 'copy this directory tree into /tmp, cd there, and run a standard <something> build command', the better.

Today you can get most of the way there with vendoring, but as I discovered this only works if you're working from within a $GOPATH. This is less than ideal because it means that the build instructions are more involved than 'cd here and run go build'. However, setting up a $GOPATH is a lot better than having to find and run an entire proxy just to satisfy vgo. A proxy makes sense if you routinely build Go programs (and running it in that case is not a big deal), but we're only likely to be building this program (or any Go program) once every few years. Adding an entire daemon that we have to run in order to do our builds would not make us happy, and even magic $GOPROXY settings would be kind of a pain (especially if we had to manually populate and maintain the cache directory).

The good news for me is that Russ Cox's posting in golang-dev is pretty much everything I want here. It appears to let me create entirely self contained directory trees (of source code, with no magic binary files) that include the full external dependencies and that can be built with a simple standard command with no setup required.

(This entry was basically overtaken by events. When Russ Cox published his series of articles, my immediate reaction was that I hated losing vendoring and I was going to object loudly in an entry. Now that the Go team has already had enough feedback to change their minds, the entry is less objecting and more trying to explain why I care about this and describe our somewhat unusual perspective on things as what I'll call 'infrequent developers', a perspective that I think is often not widely heard from.)

Comments on this page:

This was really interesting, as I'm an infrequent developer myself—I generally try to make sure sapphirecat/devproxy (a github project of mine) builds on the latest versions of Go+macOS, and other than that, I barely touch the thing. It just sits in the background, doing its job.

The simplicity of go build and the built-in cross-compiling are some of the most compelling features of the language. If they take away the simplicity, I would think about rebuilding devproxy in Python.

I'm bitter about the way vendoring has evolved over time in go, but at least they're continuing to improve it, I guess.

By cks at 2018-03-02 13:31:20:

I think that the future 'vgo' replacement will still provide the same simplicity of go build and cross compiling, especially with this vendoring option.

If I'm understanding it right, in the basic vgo world you would normally stick with a fixed version of any outside packages that you use but you aren't necessarily guaranteed that they will always be available (just as you aren't guaranteed that you can go get them today, because the upstream repository might get removed). If you use the vgo equivalent of vendoring, you guarantee availability as well, both on your normal machine and on any machine that you copy your source tree to.

(If you don't use any outside packages, vgo will presumably behave just the same as go get and go build today.)

Written on 02 March 2018.
« egrep's -o argument is great for extracting unusual fields
Frequent versus infrequent developers (in languages and so on) »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Mar 2 01:47:16 2018
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.