"Interface smuggling", a Go design pattern for expanding APIs

July 8, 2020

Interfaces are one of the big ways of creating and defining APIs in Go. Go famously encourages these interfaces to be very minimal; the widely used and implemented io.Reader and io.Writer are each one method. Minimal APIs such as this have the advantage that almost anything can implement them, which means that Go code that accepts an io.Reader or io.Writer can work transparently with a huge range of data sources and destinations.

However, this very simplicity and generality means that these APIs are not necessarily the most efficient way to perform operations. For example, if you want to copy from an io.Reader to an io.Writer, such as io.Copy() does, using only the basic API means that you have to perform intermediate data shuffling when in many cases either the source could directly write to the destination or the destination could directly read from the source. Go's solution to this is what I will call interface smuggling.

In interface smuggling, the actual implementation is augmented with additional well known APIs, such as io.ReaderFrom and io.WriterTo. Functions that want to work more efficiently when possible, such as io.Copy(), attempt to convert the io.Reader or io.Writer they obtained to the relevant API and then use it if the conversion succeeded:

if wt, ok := src.(WriterTo); ok {
   return wt.WriteTo(dst)
if rt, ok := dst.(ReaderFrom); ok {
   return rt.ReadFrom(src)
[... do copy ourselves ...]

I call this interface smuggling because we are effectively smuggling a different, more powerful, and broader API through a limited one. In the case of types supporting io.WriterTo and io.ReaderFrom, io.Copy completely bypasses the nominal API; the .Read() and .Write() methods are never actually used, at least directly by io.Copy (they may be used by the specific implementations of .WriteTo() or .ReadFrom(), or more interface smuggling may take place).

(Go code also sometimes peeks at the concrete types of interface API arguments. This is how under the right circumstances, io.Copy will wind up using the Linux splice(2) or sendfile(2) system calls.)

There is also interface smuggling that expands the API, as seen in things like io.ReadCloser and io.ReadWriteSeeker. If you have a basic io.Reader, you can try to convert it to see if it actually supports these expanded APIs, and then use them if it does.

PS: There's probably a canonical Go term for doing this as part of your API design, either to begin with or as part of expanding it while retaining backward compatibility. If so, feel free to let me know in the comments.

Comments on this page:

By Filippo Valsorda at 2020-07-09 09:46:15:

These are commonly called interface upgrades.

By cks at 2020-07-09 19:10:23:

As an additional data point, over on Twitter Michael Stapelberg called these "optional interfaces".

Written on 08 July 2020.
« Some thoughts on Fedora moving to btrfs as the default desktop file system
Link: Mime type associations (on Linux) »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Jul 8 23:37:47 2020
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.