Go's 'range over functions' iterators and avoiding iteration errors

June 17, 2024

Go is working on allowing people to range-over function iterators, and currently this is scheduled to be in Go 1.23, due out this summer (see issue 61405 and issue 61897). The actual implementation is somewhat baroque and some people have been unhappy about that (for example). My view is that this is about bringing user-written container types closer to parity with the special language container types, but recently another view of this occurred to me.

As people have noted, what is most special about this proposal is not that it creates an officially supported iteration protocol in Go, but that this protocol gets direct language support. The compiler itself will transform 'for ... = range afunction' into different code that actively implements the iteration protocol that the Go developers have chosen. This direct language support is critical to making ranging over functions like 'for k,v := range map', but it also does another thing, which is that it implements all of the details of the iteration protocol for the person writing the 'for' loop.

(People seem to generally envision that the actual usage will be 'for ... = range generator(...)', where 'generator()' is a function that returns the actual function that is used for iteration. But I think you could use method values in some situations.)

Iteration protocols are generally fairly complicated. They have to deal with setup, finalization, early exits from process of iteration, finalization in the face of early exits, and so on. The actual implementations of these protocols tends to be gnarly and somewhat subtle, with various potential mistakes and omissions that can be made, and some of these will not manifest in clear bugs until some special situation arises. Go could make everyone who wanted to use 'iterate over a function or special data structure' write out the explicit code needed to do this using the protocol, but if it did we know what the result would be; some of that code would be buggy and incomplete.

By embedding its chosen iteration protocol into the language itself, Go insures that most of that code won't have to be written by you (or by any of the plenty of people who might use user-written types and want to iterate over them). The compiler itself will take a straightforward 'for ... range' block and transform it to correctly and completely implement the protocol. In fact, the protocol is not even particularly accessible to you within the 'for' block you're writing.

People writing the iterator functions for their user-written types will have to care about the protocol, of course (although the Go protocol seems relatively simple in that regard too). But there are likely to be many fewer such iterator creators than there will be iterator users, much as Go assumes that there will be many more people using generic types than people creating them.

Written on 17 June 2024.
« Understanding a Python closure oddity
Some things on how ZFS System Attributes are stored »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jun 17 23:10:08 2024
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.