Some more feelings on nondeterministic garbage collection

April 30, 2017

A while back I wrote an entry about the problem with nondeterministic garbage collection, more or less as part of my views at the time on PyPy. In that entry I was fairly down on nondeterministic GC. I still feel more or less that way about PyPy's garbage collection. Yet at the same time I use and like Go (and I did back then), which very definitely has nondeterministic garbage collection, and I don't find it to be a problem or something that annoys me. When I was revisiting this recently, I found myself wondering what the difference is. Is it just that I like Go enough that I'm unconsciously forgiving it this?

I don't think it's that simple. Instead I think it comes down to what I could call the culture of the language but instead is better described as 'how people write code in practice'. CPython has always had a deterministic garbage collector with prompt garbage collection, and as a result people wrote plenty of code that assumes that behavior and will do various degrees of unfortunate things if it's run in an environment, like PyPy, that violates that assumption. In practice Python programmers have developed and routinely use plenty of idioms that more or less assume deterministic GC; this code may be 'incorrect' in some sense, but it's also common and normal.

(It is correct code for CPython in practice, in that it works and is efficient to write and so on.)

By contrast, Go had nondeterministic GC from the beginning and people have been coding with that in mind from the start. One partial consequence of this is that Go APIs are often carefully designed so that you can mostly avoid allocations if you want to go to the effort, with caller-supplied reusable buffers and so on. Writing such code is even pretty natural and obvious in Go, in a way that it isn't in Python. I'm pretty sure that Go's features, APIs, and coding style have all been shaped by it having nondeterministic GC, in ways that hasn't happened for Python because CPython had deterministic GC.

I also suspect that nondeterministic GC simply works better in a language that's explicitly designed to create less memory and object churn. Go has any number of language and compiler features that are partly designed to reduce memory pressure, things like unboxed array members, unboxed variables in general, and escape analysis (to enable cheap stack allocation of values).

(Static typing helps here too, but that's something that has reasons well beyond reducing memory pressure.)

PS: I don't have any directly comparable programs, but in operation this Go program seems to have about the same memory usage as this Python program, based on RSS. They aren't seeing the same load and don't quite do the same thing, but they're as close as I can get unless I get very energetic and rewrite DWiki in Go.

Comments on this page:

By lilydjwg at 2017-05-01 01:36:39:

In my opinion, Go is more like C. It tries to be efficient at the beginning, while Python focuses on easy-of-use. So you don't need to provide buffers in Python, and so your Python programs run slowly and consume lots of memory.

I think Go is mainly a better C, and Python a better shell (or Awk / Tcl / Perl) script.

Written on 30 April 2017.
« Do we want to standardize the size of our root filesystems on servers?
When a TLS client's certificate is offered to the TLS server »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Apr 30 23:09:44 2017
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.