Wandering Thoughts archives

2018-09-07

My view of the current state of Go's dependency management (as of Go 1.11)

Over on Mastodon, someone I follow was curious about the state of Go's dependency management and I had some opinions. Rather than just leave them on Mastodon, I'm going to also put them here with a bunch of additional commentary and clarifications of my toot-forced terseness.

I started with a rundown of what I see as the current positives (+) and negatives (-):

I would say Go is at about the 50% level, depending on what you want from dependency management:

+ you can lock & build with specific versions of dependencies (whether or not they've bought into versioning)
+ some packages are publishing a go.mod now (the Go version of Cargo.toml)

- lots of packages with no go.mod yet
- no crates.io equivalent
- can't just 'go get' a program with locked versions
- some other rough edges still
- the whole thing is officially still 'preliminary'.

(I mentioned Cargo.toml and crates.io because the person I was responding to is familiar with Rust.)

If what you want is 'I can make reproducible, stable versions of my own software', then I think dependency management is basically there, but you're going to be seeing a lot of git hashes as 'version numbers'. The tooling is mostly there, but the strong ecology around it has not yet formed and probably won't be for a while (Go 1.11 itself will take time to propagate around the ecology, and that's the starting point).

You can use Go 1.11's module versioning today to get stable and reproducible builds for your own packages, with controllable and easily reversible upgrades of your dependencies. However your dependencies are generally not yet versioned themselves, so what you're really doing is locking yourself to specific VCS commit identifiers (usually git hashes, because most Go packages seem to use git). With no version numbers (semantic or otherwise), upgrading dependencies is sort of a shot in the dark and you never really know what you're getting unless you actively look at the dependency in question.

Some Go packages have started to turn themselves into modules by publishing a go.mod file. However, this is not yet very common. At the moment I have 170 separate repos under $HOME/go/src ('go list' expands this to 900 packages, but many of those are internal), and only 14 of them (from 8 different authors) have go.mod files. With so little use of go.mod so far, module versioning is mostly about freezing your build and not very much about automatically giving you a 'known to work' set of dependent package versions. If you want to use package A and package A uses package B, you're mostly going to be hoping that the current git version of package A works with the current git version of package B (and then freezing those git commits). This is in contrast to a fully developed dependency management ecology (such as you have in Rust), where package A would tell you what version of package B it needed.

What I meant about go-get'ing programs is that currently there's no way to run 'go get <program>' and have this respect the program's go.mod, if it has one (as I sort of mentioned in this entry). Many people won't care about this, but if you want to distribute a program and have people build it with the specific dependent package versions you've set in your go.mod, well, it's possible right now but you have to tell people to clone your repo and then run 'go build' inside it, instead of the more convenient 'go get <myprogram>'. This is a known issue but it's not likely to be fixed before Go 1.12.

Crates.io is the central registry for the Rust package ecosystem. Go has no equivalent; the closest for finding packages is probably godoc.org or Go-Search, but then you get to pull the packages from wherever they live right now. If you want convenient one-stop shopping for packages that do <X> to use in your own code, Go doesn't have a good story for that right now. Instead you're left to do your own research and to try to sort through the various markers of quality and usage that you can find on godoc.org and Go-Search. Often this means that there is no clear, obvious, easily found community consensus choice for doing a particular job; instead information on 'you want to use <X> to do <Y>' spreads through casual conversation, blog posts, and superstition.

(An extreme example of this is getopt replacements. Everyone has their own favorite, eg.)

All of this is why I say that Go dependency management is at about the 50% level. The raw mechanics are mostly present, with some limitations, but the ecology is not, both at the low level of having versioned packages and packages with go.mods, and at the high level of having a coherent, organized package ecosystem instead of the current ad-hoc clutter. Go is certainly not at the level of Rust, where you can go to crates.io to search for a readline library and wind up on rustyline in short order, with a whole bunch of useful information in one spot.

None of this is particularly surprising. Go 1.11 itself is only a few weeks old, so it would be a little bit startling if there was already a wholesale adoption of go.mod, package versioning, and so on across the Go package world. Many people probably haven't even upgraded to Go 1.11 itself yet. Give Go's module versioning a year or two and then we can see where we are.

(The issues around a central registry are complicated, and I'm tempted to say that you probably need to build this into your cultural environment almost from the start. I'm thus not sure Go will ever be able to grow one, at least not without a fair amount of trauma and community drama.)

programming/Go1.11VersioningViews written at 23:39:19; Add Comment

I've slowly been improving my web experience by trusting uMatrix more

I have a long history of not running Javascript, which comes in large part from me very much not trusting sites that wanted to run Javascript; I started on the web in an era where Javascript was mostly abused, and that has stuck with me ever since. For most of that history the only way I had to deal with Javascript was a total block, first globally and then on a site by site basis (with NoScript). This has left me with a well honed and deep seated reflex of saying 'ha ha no' to sites that wanted to run Javascript, at least in my main Firefox browser. Basically I drifted into a mindset of declaring that it wasn't absolutely essential to enable Javascript, so I wasn't going to.

Then I switched to uMatrix for Javascript blocking, and discovered that uMatrix is a better fit for Javascript on the modern web. The big thing that uMatrix has is that it allows me to be extremely selective by only enabling 'first party' Javascript for a site, or to put it another way I can enable that site's Javascript only when I'm actively visiting it. I knew all about this back in February, but I didn't really do anything with that knowledge; I enabled first party JS for a few sites, and left it at that. After all, things had been working okay before uMatrix and I'm a creature of inertia.

(Keeping Javascript off is more than just privacy and so on; for a long time, allowing Javascript to run has been a great way to bloat up your browser's memory usage and slow it down. I don't want to let sites do that to my browser, because I want to leave my main browser session running for weeks.)

Sometimes being a creature of inertia is kind of the wrong choice. What I've slowly come to realize and convince myself of over the past while is that I don't have to be so reflexively stringent about (still) blocking this first party Javascript. Much like my slowly growing willingness to use uBlock Origin's capabilities to improve my life (1, 2), I've slowly accepted that enabling Javascript on various sites is both harmless and a genuine improvement to my life. I'm still quite selective and limited (far more so than most people would be), but I'm slowly adding more and more sites that have various bits of Javascript enabled.

It remains not entirely easy to persuade myself to enable Javascript for a site; there's still a little voice that asks me 'do you really need this? are you sure it's going to be okay?'. But I'm getting better at it, at moving past my old habits and reflexes to take advantage of uMatrix's power and possibilities to make my life nicer. I admit that part of this is simply remembering that oh yeah, I can actually do that these days; I can selectively turn on Javascript for, say, Go's code review site, and it's not a big deal any more.

(I have an entire separate browser environment for Javascript-heavy websites and it's not too hard to switch to viewing a site in that environment. But it is some extra steps, so having things working in my main browser is nicer.)

There is still a part of me that's grumpy when people's blogs or text websites partly don't work without Javascript for no particularly good reason, or at least no particularly visible one. I still partly resent needing to use Javascript to read content, and part of the reason I don't enable Javascript more broadly is a bit of stubbornness (I tell myself that I don't want to encourage people to use Javascript by having it work in my browser, but I'm aware that that's tilting at windmills at this point).

PS: Medium-based sites are not among the ones that get permanent Javascript permissions. I don't really trust Medium's Javascript much, although I tolerate it some of the time because limited Javascript in my main browser is better overall than unlimited Javascript in the 'just make it work' browser.

(This entry is partly a nudge to myself to remember that this is a perfectly viable possibility now. It works, it's easy, it doesn't seem to blow up my browser, and so on.)

web/UMatrixFirstPartyJS written at 01:41:56; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.