Wandering Thoughts archives

Making a Go program build with Go modules can be not a small change

In theory, at some point in the future Go will stop supporting the traditional GOPATH mode. When this happens, if you want to still build old Go programs that you have sitting around in checked out version control repositories, you will need to modularize them. Once upon a time, I thought that this would be as simple as going to the root of your copy of the repo, then running 'go mod init ...' and 'go mod tidy'. Unfortunately, life is not this simple and there can be at least two complications.

The first complication is moved and renamed repositories for modules, if the moved module has a go.mod that declares its new name. For example what is now github.com/hexops/vecty was once github.com/gopherjs/vecty. In a non-modular Go build, you can still import it under the old path and it will work. However, the moment you attempt to modularize the program, 'go mod tidy' will complain and stop:

github.com/gopherjs/vecty: github.com/gopherjs/vecty@v0.6.0: parsing go.mod:
module declares its path as: github.com/hexops/vecty
        but was required as: github.com/gopherjs/vecty

In theory you may be able to get this to work with a go.mod replace directive. In practice my attempts to do this resulted in 'go mod tidy' errors about:

go: github.com/hexops/vecty@v0.6.0 used for two different module paths (github.com/gopherjs/vecty and github.com/hexops/vecty)

(You also need to get the version number or other version identifier of the moved repository.)

The general fix is to edit every import of packages from the module to use the new location. Then you can run 'go mod tidy' without it complaining.

The second complication is modules that have moved to versions above v1, possibly very far past v1; for example, github.com/google/go-github is up to v37, and modularized at v18 (it doesn't even have a tagged v1). A GOPATH build of the program you're trying to modularize will use whatever version of the repository you have checked out, which may well be the current one, and the code will import it as a version without a version suffix (as 'github.com/google/go-github'). When you run 'go mod tidy', Go will attempt to find the most recent tag (or version of the repository) that doesn't have a go.mod file, and specify that version in your go.mod with a '+incompatible' tag. Depending on how far Go had to rewind, this may be a version of the package that is far older than the program expects.

(If a go.mod existed for a v1 version, I suspect that 'go mod tidy' will pick that in this case. But I haven't tried to test it, partly for lack of a suitable module to test against. With github.com/google/go-github, I get 'v17.0.0+incompatible', the last tagged version before it was modularized.)

Again the fix is to edit the program's source code to change every import of the package to use the proper versioned package. Instead of importing, say, 'github.com/google/go-github/github', you would import 'github.com/google/go-github/v37/github'.

Although I haven't tested it extensively, it appears that go-imports-rename can be used to make both sorts of changes. I successfully used it to automatically modify my test third party repository.

(There may be other tools to do this package import renaming, but this is the one I could find.)

The unfortunate part of all of this is that it requires you to make changes to files that will be under version control in the repo. If the upstream updates things in the future, this will probably make your life more complicated.

(In some cases, 'go mod tidy' may insist that you clean up imports in code that's in sub-packages in the repository that aren't actually imported and used in the program itself.)