2024-02-24
The Go 'range over functions' proposal and user-written container types
In Go 1.22, the Go developers have made available a "range over function" experiment, as described in the Go Wiki's "Rangefunc Experiment". Recently I read a criticism of this, Richard Ulmer's Questioning Go's range-over-func Proposal (via). As I read Ulmer's article, it questions the utility of the range over func (proposed) feature based on the grounds that this isn't a significant enough improvement in standard library functions like strings.Split (which is given as an example in the "more motivation" section of the wiki article).
I'm not unsympathetic to this criticism, especially when it concerns standard library functionality. If the Go developers want to extend various parts of the standard library to support streaming their results instead of providing the results all at once, then there may well be better, lower-impact ways of doing so, such as developing a standard API approach or set of approaches for this and then using this to add new APIs. However, I think that extending the standard library into streaming APIs is by far the less important side of the "range over func" proposal (although this is what the "more motivation" section of the wiki article devotes the most space to).
Right from the beginning, one of the criticisms of Go was that it had some privileged, complex builtin types that couldn't be built using normal Go facilities, such as maps. Generics have made it mostly possible to do equivalents of these (generic) types yourself at the language level (although the Go compiler still uniquely privileges maps and other builtin types at the implementation level). However, these complex builtin types still retain some important special privileges in the language, and one of them is that they were the only types that you could write convenient 'range' based for loops.
In Go today you can write, for example, a set type or a key/value type with some complex internal storage implementation and make it work even for user-provided element types (through generics). But people using your new container types cannot write 'for elem := range set' or 'for k, v := range kvstore'. The best you can give them is an explicit push or pull based iterator based on your type (in a push iterator, you provide a callback function that is given each value; in a pull iterator, you repeatedly call some function to obtain the next value). The "range over func" proposal bridges this divide, allowing non-builtin types to be ranged over almost as easily as builtin types. You would be able to write types that let people write 'for elem := range set.Forward()' or 'for k, v := kvstore.Walk()'.
This is an issue that can't really be solved without language support. You could define a standard API for iterators and iteration (and the 'iter' package covered in the wiki article sort of is that), but it would still be more code and somewhat awkward code for people using your types to write. People are significantly attracted to what is easy to program; the more difficult it is to iterate user types compared to builtin types, the less people will do it (and the more they will use builtin types even when they aren't a good fit). If Go wants to put user (generic) types on almost the same level (in the language) as builtin types, then I feel it needs some version of a "range over func" approach.
(Of course, you may feel that Go should not prioritize putting user types on almost the same level as builtin types.)