One of the common models for handling asynchronous IO is what I'll call the queued IO model, where you put all of your IO operations in some sort of a queue and then as things become ready, the OS completes various ones and hands them back to you. Sometimes this queue is explicitly exposed and sometimes, as in Go, the queue is implicit in a collection of threads all doing (what they see as) blocking IO operations. The queued IO model is generally simple and attractive, either in threaded form (in Go) or in explicit form where you pass operations that you'd like to do to the OS and it notifies you when various ones finish.

Recently I wound up reading Evan Klitzke's Goroutines, Nonblocking I/O, And Memory Usage, which pointed out a drawback to this model that hadn't been obvious to me before. That drawback is memory usage for pending operations, especially reads, in a situation where you have a significant number of idle connections. Suppose that you have 1,000 connections where you're waiting for the client to send you something. In a queued IO model the normal way to operate is to queue 1,000 read operations, and each of these queued read operations must come with an allocated buffer for the operating system to write the read data into. If only (say) 5% of those connections are active at any one time, you have quite a lot of memory tied up in buffers that are just sitting around inactive. In a select() style model that exposes readiness before you perform the IO, you can only allocate buffers when you're actually about to read data.

Writes often pre-compute and pre-allocate the data to be written, in which case this isn't much of an issue for them; the buffer for the data to be written has to be allocated beforehand either way. But in situations where the data to be written could be generated lazily on the fly, the queued IO model can once again force extra memory allocations where you have to allocate and fill buffers for everything, not just the connections that are ready to have more data pushed to them.

All of this may be obvious to people already, but it was surprising to me so I feel like writing it down, especially how it extends from Go style 'blocking IO with threads' to the general model of queuing up asynchronous IO operations for the kernel to complete for you as it can.

(Of course there are reasons to want a select() like interface beyond this issue, such as the cancellation problem.)