Using Go finalizers can be a better option than not using them

April 6, 2018

Go has finalizers, which let you have some code be invoked just as an object is about to be garbage collected. However, plenty of people don't like them and the usual advice is to completely avoid them (for example). Recently, David Crawshaw wrote The Tragedy of Finalizers (via), in which he points out various drawbacks of finalizers and shows a case where relying on them causes failures. I more or less agree with all of this, but at the same time, I've used finalizers myself in a Go package to access to Solaris/Illumos kstats and I'll defend that usage.

What I use finalizers for is to avoid an invisible leak if people don't use my API correctly. In theory when you call my package you get back a magic token, which holds the only reference to some C-allocated memory. When you're done with the token, you're supposed to call a method to close it down, which will free the C-allocated memory. In practice, well, people make API usage and object lifetime mistakes. Without a finalizer, if a token went out of scope and was lost to garbage collection we'd permanently leak that C-allocated memory. As with all memory and resource leaks of this nature, this would be an especially annoying and pernicious leak because it would be completely invisible from the Go level. None of the usual Go level memory leak tools would help you at all (and I suspect that the usual C leak finding tools would have serious problems due to the presence of Go).

At one level, using a finalizer here is a pragmatic decision; it protects people using my package from certain usage errors that would cause problems that are hard to deal with. At another level, though, I can argue that using finalizers here is actually within the broad spirit of Go. As a garbage collected language, Go has essentially made a decision that explicitly managing object lifetimes is too hard, too much work, and too error-prone. It's a bit peculiar to be perfectly fine with this for memory, but not fine with this for other resources for anything other than purely pragmatic reasons.

(At the same time, those pragmatic reasons are real; as David Crawshaw explains, relying on memory garbage collection to garbage collect other resources before you run out of them is at best dangerous. Even my case is a bit dubious, since C-allocated memory doesn't apply pressure to the Go garbage collector.)

David Crawshaw followed up his article with Sharp-Edged Finalizers in Go, where he advocated using finalizers in this situation to force panics when people fail to use your APIs correctly. You can do this, but it feels somewhat un-Go-like to me. As a result I think you should only resort to this if the consequence of not using your API correctly are quite severe (for example, potential data loss because you forgot to commit a database transaction and then check for errors in it).

As a general note, I wouldn't say that my sort of use of finalizers is intended to avoid resource leaks as such. You will have a resource leak in practice from the time when you stop needing the resource (the kstat token, the open file, or what have you) until the Go garbage collection calls your finalizer (if it ever does), because the resource is still there but neither in use nor wanted. What finalizers do is make that leak be theoretically a temporary one, instead of definitely permanent. In other words, it's a recoverable leak instead of an unrecoverable one.

PS: This isn't original to me, of course. For example, this unofficial Go FAQ says that this is the main use of finalizers, and there's the example of the *os.File finalizer in the standard library.

(This has been on my mind for a while, but David Crawshaw's articles provide a convenient prompt and I hadn't thought of using finalizers to force a hard error in this situation.)

Comments on this page:

By Aneurin Price at 2018-04-06 09:54:21:

You will have a resource leak in practice from the time when you stop needing the resource (the kstat token, the open file, or what have you) until the Go garbage collection calls your finalizer (if it ever does)

So there's no deterministic way to execute some code when an object goes out of scope? Does Go at least have something like Python's "with" statement? If not, it feels like any API implemented in Go is doomed to hit a maximum point of maybe 3 on Rusty's API scale.

I can't say I find that enormously surprising though. Everything I've ever read about Go makes it sound like it's a collection of all the worst language design mistakes of all time, all rolled into one neat package of ultimate garbage. Sort of the platonic ideal of 'worst possible programming language'.

By cks at 2018-04-06 16:18:35:

Go has defer statements that let you deterministically execute code at the end of functions, but that's it. There's no direct equivalent of Python's with and similar things.

Given Go's goals, I (currently) agree with this on the grounds that Go is being honest and explicit about what it provides and what effects it has. In a language with references, 'when an object goes out of scope' is either fuzzy or subject to misunderstandings by programmers, and Go doesn't pretend to provide deterministic and reliable 'when an object becomes inaccessible' (which would probably take something like Rust's lifetime management if you wanted it to act promptly). It might be nice to have nicer syntax and more encapsulation, but Go is kind of generally against that; it's deliberately a small and explicit language as part of its tradeoffs.

(For the goals Go was created with, I think that they are the right tradeoffs even if they make the language less interesting and more verbose.)

Python's with statement (or an equivalent in some other language) won't help with the sort of situation the article appears to be describing. Say you provide a library of API functions that return objects (or magic tokens, or whatever). You can't have those functions return within the scope of a with statement; when the function returns, the with statement's scope is exited and the __exit__ method of the context manager is invoked. So there's no way to use the with statement to manage resources that the function has to pass on to its caller.

Written on 06 April 2018.
« Switching over to Firefox Quantum was relatively painless
Some numbers for how well various compressors do with our /var/mail backup »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Apr 6 01:32:14 2018
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.