Some notes on keeping up with Go packages and commands

January 26, 2015

Bearing in mind that just go get'ing things is a bad way to remember what packages you're interested in, it can be useful to keep an eye on updates to Go packages and commands. My primary tool for this is Dmitri Shuralyov's Go-Package-Store, which lets you keep an eye on not only what stuff in $GOPATH/src has updates but what they are. However, there are a few usage notes that I've accumulated.

The first and most important thing to know about Go-Package-Store, and something that I only realized recently myself (oh the embarrassment) is that Go-Package-Store does not rebuild packages or commands. All it does is download new versions (including fetching and updating their dependencies). You can see this in the commands it's running if you pay attention, since it specifically runs 'go get -u -d'. This decision is sensible and basically necessary, since many commands and (sub) packages aren't installed with 'go get <repo top level>', but it does mean that you're going to have to do this yourself when you want to.

So, the first thing this implies is that you need to keep track of the go get command to rebuild each command in $GOPATH/bin that you care about; otherwise, sooner or later you'll be staring at a program in $GOPATH/bin and resorting to web searches to find out what repository it came from and how it's built. I suggest just putting this information in a simple shell script that just does a mass rebuild, with one 'go get' per line; when I want to rebuild just a specific command, I cut and paste its line.

(Really keen people will turn the text file into a script so that you can do things like 'rebuild <command>' to run the right 'go get' to rebuild the given command.)

The next potentially tricky area is dependent packages, in several ways. The obvious thing is that having G-P-S update a dependent package doesn't in any way tell you that you should rebuild the command that uses it; in fact G-P-S doesn't particularly know what uses what package. The easy but bruce force way to deal with this is just to rebuild all commands every so often (well, run 'go get -u' against them, I'm not sure how much Make-like dependency checking it does).

The next issue is package growth. What I've noticed over time is that using G-P-S winds up with me having extra packages that aren't needed by the commands (and packages) that I have installed. As a result I both pay attention to what packages G-P-S is presenting updates for and periodically look through $GOPATH/src for packages that make me go 'huh?'. Out of place packages get deleted instead of updated, on the grounds that if they're actual dependencies of something I care about they'll get re-fetched when I rebuild commands.

(I also delete $GOPATH/pkg/* every so often. One reason that all of this rebuilding doesn't bother me very much is that I track the development version of Go itself, so I actively want to periodically rebuild everything with the latest compiler. People with big code bases and stable compilers may not be so sanguine about routinely deleting compiled packages and so on.)

I think that an explicit 'go get -u' of commands and packages that you care about will reliably rebuild dependent packages that have been updated but not (re)built in the past by Go-Package-Store, but I admit that I sometimes resort to brute force (ie deleting $GOPATH/pkg/*) just to be sure. Go things build very fast and I'm not building big things, so my attitude is 'why not?'.

Sidebar: Where I think the extra packages come from

This is only a theory. I haven't tested it directly; it's just the only cause I can think of.

Suppose you have a command that imports a sub-package from a repository. When you 'go get' the command, I believe that Go only fetches the further imported dependencies of the sub-package itself. Now, later on Go-Package-Store comes along, reports that the repository is out of date, and when you tell it to update things it does a 'go get' on the entire repository (not just the sub-package initially used by the command). This full-repo 'go get' presumably imports either all dependencies used in the repository or all dependencies of the code in the top level of the repository (I'm not sure which), which may well add extra dependencies over what the sub-package needed.

(The other possible cause is shifting dependencies in packages that I use directly, but some stray packages are so persistent in their periodic returns that I don't really believe that.)

Written on 26 January 2015.
« The long term problem with ZFS on Linux is its license
Our current email anti-virus system is probably ineffective now »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jan 26 01:53:43 2015
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.