Wandering Thoughts archives

2018-09-07

My view of the current state of Go's dependency management (as of Go 1.11)

Over on Mastodon, someone I follow was curious about the state of Go's dependency management and I had some opinions. Rather than just leave them on Mastodon, I'm going to also put them here with a bunch of additional commentary and clarifications of my toot-forced terseness.

I started with a rundown of what I see as the current positives (+) and negatives (-):

I would say Go is at about the 50% level, depending on what you want from dependency management:

+ you can lock & build with specific versions of dependencies (whether or not they've bought into versioning)
+ some packages are publishing a go.mod now (the Go version of Cargo.toml)

- lots of packages with no go.mod yet
- no crates.io equivalent
- can't just 'go get' a program with locked versions
- some other rough edges still
- the whole thing is officially still 'preliminary'.

(I mentioned Cargo.toml and crates.io because the person I was responding to is familiar with Rust.)

If what you want is 'I can make reproducible, stable versions of my own software', then I think dependency management is basically there, but you're going to be seeing a lot of git hashes as 'version numbers'. The tooling is mostly there, but the strong ecology around it has not yet formed and probably won't be for a while (Go 1.11 itself will take time to propagate around the ecology, and that's the starting point).

You can use Go 1.11's module versioning today to get stable and reproducible builds for your own packages, with controllable and easily reversible upgrades of your dependencies. However your dependencies are generally not yet versioned themselves, so what you're really doing is locking yourself to specific VCS commit identifiers (usually git hashes, because most Go packages seem to use git). With no version numbers (semantic or otherwise), upgrading dependencies is sort of a shot in the dark and you never really know what you're getting unless you actively look at the dependency in question.

Some Go packages have started to turn themselves into modules by publishing a go.mod file. However, this is not yet very common. At the moment I have 170 separate repos under $HOME/go/src ('go list' expands this to 900 packages, but many of those are internal), and only 14 of them (from 8 different authors) have go.mod files. With so little use of go.mod so far, module versioning is mostly about freezing your build and not very much about automatically giving you a 'known to work' set of dependent package versions. If you want to use package A and package A uses package B, you're mostly going to be hoping that the current git version of package A works with the current git version of package B (and then freezing those git commits). This is in contrast to a fully developed dependency management ecology (such as you have in Rust), where package A would tell you what version of package B it needed.

What I meant about go-get'ing programs is that currently there's no way to run 'go get <program>' and have this respect the program's go.mod, if it has one (as I sort of mentioned in this entry). Many people won't care about this, but if you want to distribute a program and have people build it with the specific dependent package versions you've set in your go.mod, well, it's possible right now but you have to tell people to clone your repo and then run 'go build' inside it, instead of the more convenient 'go get <myprogram>'. This is a known issue but it's not likely to be fixed before Go 1.12.

Crates.io is the central registry for the Rust package ecosystem. Go has no equivalent; the closest for finding packages is probably godoc.org or Go-Search, but then you get to pull the packages from wherever they live right now. If you want convenient one-stop shopping for packages that do <X> to use in your own code, Go doesn't have a good story for that right now. Instead you're left to do your own research and to try to sort through the various markers of quality and usage that you can find on godoc.org and Go-Search. Often this means that there is no clear, obvious, easily found community consensus choice for doing a particular job; instead information on 'you want to use <X> to do <Y>' spreads through casual conversation, blog posts, and superstition.

(An extreme example of this is getopt replacements. Everyone has their own favorite, eg.)

All of this is why I say that Go dependency management is at about the 50% level. The raw mechanics are mostly present, with some limitations, but the ecology is not, both at the low level of having versioned packages and packages with go.mods, and at the high level of having a coherent, organized package ecosystem instead of the current ad-hoc clutter. Go is certainly not at the level of Rust, where you can go to crates.io to search for a readline library and wind up on rustyline in short order, with a whole bunch of useful information in one spot.

None of this is particularly surprising. Go 1.11 itself is only a few weeks old, so it would be a little bit startling if there was already a wholesale adoption of go.mod, package versioning, and so on across the Go package world. Many people probably haven't even upgraded to Go 1.11 itself yet. Give Go's module versioning a year or two and then we can see where we are.

(The issues around a central registry are complicated, and I'm tempted to say that you probably need to build this into your cultural environment almost from the start. I'm thus not sure Go will ever be able to grow one, at least not without a fair amount of trauma and community drama.)

Go1.11VersioningViews written at 23:39:19; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.