Making your own changes to things that use Go modules

September 11, 2019

Suppose, not hypothetically, that you have found a useful Go program but when you test it you discover that it has a bug that's a problem for you, and that after you dig into the bug you discover that the problem is actually in a separate package that the program uses. You would like to try to diagnose and fix the bug, at least for your own uses, which requires hacking around in that second package.

In a non-module environment, how you do this is relatively straightforward, although not necessarily elegant. Since building programs just uses what's found in in $GOPATH/src, you can cd directly into your local clone of the second package and start hacking away. If you need to make a pull request, you can create a branch, fork the repo on Github or whatever, add your new fork as an additional remote, and then push your branch to it. If you didn't want to contaminate your main $GOPATH with your changes to the upstream (since they'd be visible to everything you built that used that package), you could work in a separate directory hierarchy and set your $GOPATH when you were working on it.

If the program has been migrated to Go modules, things are not quite as straightforward. You probably don't have a clone of the second package in your $GOPATH, and even if you do, any changes to it will be ignored when you rebuild the program (if you do it in a module-aware way). Instead, you make local changes by using the 'replace' directive of the program's go.mod, and in some ways it's better than the non-module approach.

First you need local clones of both packages. These clones can be a direct clone of the upstream or they can be clones of Github (or Gitlab or etc) forks that you've made. Then, in the program's module, you want to change go.mod to point the second package to your local copy of its repo:

replace github.com/rjeczalik/which => /u/cks/src/scratch/which

You can edit this in directly (as I did when I was working on this) or you can use 'go mod edit'.

If the second package has not been migrated to Go modules, you need to create a go.mod in your local clone (the Go documentation will tell you this if you read all of it). Contrary to what I initially thought, this new go.mod does not need to have the module name of the package you're replacing, but it will probably be most convenient if it does claim to be, eg, github.com/rjeczalik/which, because this means that any commands or tests it has that import the module will use your hacks, instead of quietly building against the unchanged official version (again, assuming that you build them in a module-aware way).

(You don't need a replace line in the second package's go.mod; Go's module handling is smart enough to get this right.)

As an important note, as of Go 1.13 you must do 'go get' to build and install commands from inside this source tree even if it's under $GOPATH. If it's under $GOPATH and you do 'go get <blah>/cmd/gobin', Go does a non-module 'go get' even though the directory tree has a go.mod file and this will use the official version of the second package, not your replacement. This is documented but perhaps surprising.

When you're replacing with a local directory this way, you don't need to commit your changes in the VCS before building the program; in fact, I don't think you even need the directory tree to be a VCS repository. For better or worse, building the program will use the current state of your directory tree (well, both trees), whatever that is.

If you want to see what your module-based binaries were actually built with in order to verify that they're actually using your modified local version, the best tool for this is 'go version -m'. This will show you something like:

go/bin/gobin go1.13
  path github.com/rjeczalik/bin/cmd/gobin
  mod  github.com/rjeczalik/bin    (devel)
  dep  github.com/rjeczalik/which  v0.0.0-2014[...]
  =>    /u/cks/go/src/github.com/siebenmann/which

I believe that the '(devel)' appears if the binary was built directly from inside a source tree, and the '=>' is showing a 'replace' in action. If you build one of the second package's commands (from inside its source tree), 'go version -m' doesn't report the replacement, just that it's a '(devel)' of the module.

(Note that this output doesn't tell us anything about the version of the second package that was actually used to build the binary, except that it was the current state of the filesystem as of the build. The 'v0.0.0-2014[...]' version stamp is for the original version, not our replacement, and comes from the first package's go.mod.)

PS: If 'go version -m' merely reports the 'go1.13' bit, you managed to build the program in a non module-aware way.

Sidebar: Replacing with another repo instead of a directory tree

The syntax for this uses your alternate repository, and I believe it must have some form of version identifier. This version identifier can be a branch, or at least it can start out as a branch in your go.mod, so it looks like this:

replace github.com/rjeczalik/which => github.com/siebenmann/which reliable-find

After you run 'go build' or the like, the go command will quietly rewrite this to refer to the specific current commit on that branch. If you push up a new version of your changes, you need to re-edit your go.mod to say 'reliable-find' or 'master' or the like again.

Your upstream repository doesn't have to have a go.mod file, unlike the case with a local directory tree. If it does have a go.mod, I think that the claimed package name can be relatively liberal (for instance, I think it can be the module that you're replacing). However, some experimentation with sticking in random upstreams suggests that you want the final component of the module name to match (eg, '<something>/which' in my case).


Comments on this page:

By Greg A. Woods at 2019-09-15 12:35:23:

Thanks for the writeup! This should save me some time!

Written on 11 September 2019.
« Catching Control-C and a gotcha with shell scripts
The mystery of why my Fedora 30 office workstation was booting fine »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Sep 11 20:49:02 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.