Link: Linux Load Averages: Solving the Mystery
Load averages are an industry-critical metric – my company spends millions auto-scaling cloud instances based on them and other metrics – but on Linux there's some mystery around them. Linux load averages track not just runnable tasks, but also tasks in the uninterruptible sleep state. Why? I've never seen an explanation. In this post I'll solve this mystery, and summarize load averages as a reference for everyone trying to interpret them.
In the process of doing this, Brendan Gregg goes back to TENEX (including its source code) for the more or less original load average. Then he chases down the kernel patch from October 1993 that changed Linux's load averages from purely based on the size of the run queue to including processes in disk wait. It goes on from there, including some great examples of how to break down a load average to see what's contributing what (using modern Linux tracing tools, which Gregg is an expert on). The whole thing is really impressive and worth reading.
(Gregg's discussion is focused on Linux alone. For a cross-Unix view, I've written entries on when the load average was added to Unix and the many load averages of different Unix strains. In the latter entry I confidently asserted that Linux's load average included 'disk wait' processes from the start, which Gregg's research has revealed to be wrong.)
Link: How does "the" X11 clipboard work?
X11: How does “the” clipboard work? (via) is a technical walk through the modern X11 selection system, one that winds up discussing things at the level of the X protocol and Xlib, with helpful code examples. I learned some quite useful things in the process, for example how to use xclip to query things to find out what formats a selection is available in.
(Technical details about X selections are relevant to me because I use a program that deals with them and which I'd like to see do so more conveniently.)
Link: NASA DASHlink - Real System Failures
The Observed Failures slide deck from NASA DASHlink (via many places) is an interesting and even alarming collection of observed failures in hardware and software, mostly avionics related. I find it both entertaining and a useful reminder that all of this digital stuff is really analog underneath and that leads to interesting failure modes. Lest you think that all of these were hardware faults and us software people can be smug, well, not really. There are more; read the whole thing, as they say.
Link: ZFS Storage Overhead
ZFS Storage Overhead (via) is not quite about what you might think. It's not about, say, the overhead added by ZFS's RAIDZ storage (where there are surprises); instead it's about some interesting low level issues of where space disappears to even in very simple pools. The bit about metaslabs was especially interesting to me. It goes well with Matthew Ahrens' classic ZFS RAIDZ stripe width, or: How I Learned to Stop Worrying and Love RAIDZ, which is endlessly linked and cited for very good reasons.
Link: Survey of [floating point] Rounding Implementations in Go
Rounding in Go is hard to do correctly. That is, given a
float64, truncate the fractional part (anything right of the decimal point), and add one to the truncated value if the fractional part was >= 0.5. [...]
But it's a lot more than that, because the article then proceeds
to demonstrate just how complicated floating point rounding really
is (in any language), how non-obvious this complexity is, and how
easy it is to get it wrong. It ends with a walk-through of what
math.Round in Go 1.10, which is implemented not through
floating point operations but through direct knowledge of how
floating point values are implemented at
the bit level.
Floating point is one of those things in programming that looks like it's reasonably simple but instead is a deep pit of potential complexity once you move beyond doing very simple things. I enjoy articles like this because they are both a good reminder of this and a peek behind the curtain.
Links: Git remote branches and Git's missing terminology (and more)
Mark Jason Dominus's Git remote branches and Git's missing terminology (via) is about what it sounds like. This is an issue of some interest to me, since I flailed around in the same general terminology swamp in a couple of recent entries. Dominus explains all of this nicely, with diagrams, and it helped reinforce things in my mind (and reassure me that I more or less understood what was going on).
He followed this up with Git's rejected push error, which covers a
git push' issue with the same thoroughness as his first
article. I more or less knew this stuff, but again I found it useful
to read through his explanation to make sure I actually knew as much
as I thought I did.
Link: The evolution of Unix's overall architecture
Diomidis Spinellis has created a set of great resources for looking at Unix's history. He started with the Unix history rep and has then used that to create block diagrams of Unix's structure in V1 and modern FreeBSD. His article Unix Architecture Evolution Diagrams (via) explains the interesting story of how he put these diagrams together. He also has a site on where manpages appear over Unix versions.
Link: Rob Landley's Linux Memory Management Frequently Asked Questions
Rob Landley's Linux Memory Management Frequently Asked Questions is what it says in the title (via). As of writing this pointer to it, the current version of the FAQ appears to be from 2014, but much of its information is relatively timeless.
Landley's FAQ is good for a lot more than Linux specific information. It's a good general overview of a number of hardware MMU concepts and general Unix MMU concepts. Things like page tables, TLBs, and even memory mappings are basically universal, and the FAQ has a nice solid explanation of all of them and much more.
Link: Introduction to Certificate Transparency for Server Operators
Alex Gaynor's Introduction to Certificate Transparency for Server Operators (via) is what it says in the title, and taught me some things about certificate transparency in practice. Sadly, one of the things it taught me is that once again Lighttpd seems to be coming up short as far as modern TLS goes. I really should switch over my personal site to using Apache, even if it will kind of be a pain because Fedora fumbled good Apache configuration.
(I also hadn't known about Cert Spotter, which has the advantage that you don't need a Facebook login to use it and thus don't have to helpfully (for Facebook) tie one or more domains to your Facebook login. All you need is an email address and on the modern Internet, you already need a source of individualized disposable ones.)
Link: The Unix Heritage Society now has the 8th, 9th, and 10th editions of Research Unix
Today in an email message with the subject of [TUHS] Release of 8th, 9th and 10th Editions Unix, Warren Toomey announced that the Unix Heritage Society has now gained permission to make the source code of Research Unix's 8th, 9th, and 10th editions available for the usual non-commercial purposes. This apparently is the result of a significant lobbying campaign from a variety of Unix luminaries. The actual source trees can be found in TUHS' archive area for Research distributions.
Most people are familiar with Research Unix versions through V7
(the 7th Edition), which was the famous one that really got out
into the outside world and started the Unix revolution. The 8th
through 10th editions were what happened inside Bell Labs after
this (with a trip through BSD for the port to Vaxen, among other
things; see the history of Unix), and because Unix
was starting to be commercialized around when they were being worked
on by Bell Labs, they were never released in the way that the 7th
Edition was. Despite that they were the foundation of some significant
innovations, such as the original Streams and
/proc, and for various
reasons they acquired a somewhat legendary status as the last
versions of the true original strain of Research Unix. Which
you couldn't see or run, which just added to the allure.
You probably still can't actually run these editions, unless you want to engage in extensive hardware archaeology and system (re)construction. But at least the Unix community now has these pieces of history.