Wandering Thoughts


Link: Rob Landley's Linux Memory Management Frequently Asked Questions

Rob Landley's Linux Memory Management Frequently Asked Questions is what it says in the title (via). As of writing this pointer to it, the current version of the FAQ appears to be from 2014, but much of its information is relatively timeless.

Landley's FAQ is good for a lot more than Linux specific information. It's a good general overview of a number of hardware MMU concepts and general Unix MMU concepts. Things like page tables, TLBs, and even memory mappings are basically universal, and the FAQ has a nice solid explanation of all of them and much more.

(See also Rob Landley's other writing, and his website in general.)

LandleyLinuxMMFAQ written at 20:35:13; Add Comment


Link: Introduction to Certificate Transparency for Server Operators

Alex Gaynor's Introduction to Certificate Transparency for Server Operators (via) is what it says in the title, and taught me some things about certificate transparency in practice. Sadly, one of the things it taught me is that once again Lighttpd seems to be coming up short as far as modern TLS goes. I really should switch over my personal site to using Apache, even if it will kind of be a pain because Fedora fumbled good Apache configuration.

(I also hadn't known about Cert Spotter, which has the advantage that you don't need a Facebook login to use it and thus don't have to helpfully (for Facebook) tie one or more domains to your Facebook login. All you need is an email address and on the modern Internet, you already need a source of individualized disposable ones.)

CertTransForServerOps written at 13:50:48; Add Comment


Link: The Unix Heritage Society now has the 8th, 9th, and 10th editions of Research Unix

Today in an email message with the subject of [TUHS] Release of 8th, 9th and 10th Editions Unix, Warren Toomey announced that the Unix Heritage Society has now gained permission to make the source code of Research Unix's 8th, 9th, and 10th editions available for the usual non-commercial purposes. This apparently is the result of a significant lobbying campaign from a variety of Unix luminaries. The actual source trees can be found in TUHS' archive area for Research distributions.

Most people are familiar with Research Unix versions through V7 (the 7th Edition), which was the famous one that really got out into the outside world and started the Unix revolution. The 8th through 10th editions were what happened inside Bell Labs after this (with a trip through BSD for the port to Vaxen, among other things; see the history of Unix), and because Unix was starting to be commercialized around when they were being worked on by Bell Labs, they were never released in the way that the 7th Edition was. Despite that they were the foundation of some significant innovations, such as the original Streams and /proc, and for various reasons they acquired a somewhat legendary status as the last versions of the true original strain of Research Unix. Which you couldn't see or run, which just added to the allure.

You probably still can't actually run these editions, unless you want to engage in extensive hardware archaeology and system (re)construction. But at least the Unix community now has these pieces of history.

UnixHeritageSocietyUnixV8V9V10 written at 21:50:32; Add Comment


Link: Four Column ASCII

Four Column ASCII (via) is a reformatted version of this Hacker News comment that presents the ASCII chart in four columns of 32 characters instead of the usual presentation of eight columns of 16 (as seen eg here). A four column representation makes a number of patterns in ASCII (both bit-wise and otherwise) much clearer, for example why Ctrl-Space, Ctrl-@, and Ctrl-` all generally generate a null byte.

(I suspect that ASCII is traditionally presented in an 8 column by 16 row table because that fits into an 80x24 terminal or terminal window.)

FourColumnASCII written at 19:45:41; Add Comment


Link: Let's Stop Ascribing Meaning to Code Points

Manish Goregaokar's Let's Stop Ascribing Meaning to Code Points starts out with this:

I've seen misconceptions about Unicode crop up regularly in posts discussing it. One very common misconception I've seen is that code points have cross-language intrinsic meaning.

He goes on to explain the ways that this is dangerous and how tangled this area of Unicode is. I knew little bits of this already, but apparently combining characters are only the tip of the iceberg.

(via, and see also.)

UnicodeCodePointsNoMeaning written at 16:54:18; Add Comment


Link: RFC 6919: Further Key Words for Use in RFCs to Indicate Requirement Levels

If you read RFCs, you may know about the standard use of the MUST, SHOULD, and so on key words that come from RFC 2119. RFC 6919, issued April 1st 2013, adds some useful additional key words like "MUST (BUT WE KNOW YOU WON'T)", "REALLY SHOULD NOT", and the like.

By itself this would be amusing and interesting. But what really makes RFC 6919 rewarding to read is that it shows usage examples for each of its new key words that are drawn from existing RFCs. If you have much exposure to how RFCs are actually implemented in the field, this will make you alternate laughter and sad sighs. To quote myself from when I first saw it:

RFC 6919 makes me laugh but it's sad laughter. Such honesty had to be published Monday.

(I was reminded of RFC 6919 by @whitequark's tweet, and was actually surprised to discover that I've never linked to it here on Wandering Thoughts. So now I'm fixing that.)

RFC6919MoreKeywords written at 22:11:39; Add Comment


Link: Twice the bits, twice the trouble: vulnerabilities induced by migrating to 64-bit platforms

Adrian Colyer's Twice the bits, twice the trouble: vulnerabilities induced by migrating to 64-bit platforms (via) is a very readable and very interesting summary of an ultimately depressing academic paper on the security vulnerabilities that get induced in C code simply by migrating from 32-bit platforms to 64-bit platforms.

In theory I sort of knew about all of this, but it's one thing to vaguely have heard about it and another thing to see handy comparison charts and examples and so on of how relatively innocent C code introduces real vulnerabilities simply when you rebuild it on 64-bit machines and then expose it to attackers.

Here's a depressing quote from the academic paper to finish up with and motivate reading at least Colyer's summary of the whole thing:

Finally, we make use of this systematization and the experience thus gained to uncover 6 previously unknown vulnerabilities in popular software projects, such as the Linux kernel, Chromium, the Boost C++ Libraries and the compression libraries libarchive and zlibā€”all of which have emerged from the migration from 32-bit to 64-bit platforms.

That's, well, unfortunate. But not unexpected, I suppose. Maybe all future languages should not have any limited-range numeric types that can have different sizes on different platforms, even if it's theoretically an attractive idea for 'optimization'.

(I don't know what Rust does here, but Go does have int and uint, which are either 32 or 64 bits depending on the platform.)

C64BitMigrationVulnerabilities written at 14:01:52; Add Comment


Link: Linux containers in 500 lines of code

Lizzie Dixon's Linux Containers in 500 lines of code is exactly what it says in the title, namely an example of setting up a relatively complete Linux container in 500 lines of code or so. The whole thing is interesting, but for me the really useful bits are the code examples and careful discussion of things like dropping capabilities and installing seccomp filters to block system calls.

(I feel I have a broad understanding of how Linux containers work and the general kernel pieces used to set them up, but I haven't really done anything with either capabilities or seccomp.)

Also, it turns out that reading the footnotes here was very useful because it led to me discovering the Ubuntu sysctl kernel.unprivileged_userns_clone, which lets you turn off unprivileged use of user namespaces. I've wanted this for some time so it's great to find out that we can get it on most of our Ubuntu machines.

LinuxContainers500Lines written at 13:15:27; Add Comment


Link: Actually using ed

Tom Ryder's Actually using ed (via @davecheney) is a nice little walk-through of using the Unix ed editor to, well, edit some text. I've used ed a long time ago, and this inspired me to fire it up again to follow along and play around a bit; it was surprising how much I remembered. It was also nice reading about some advanced ed features that I either didn't know about or never used enough to remember.

(As always, I find it interesting what's considered basic versus advanced in things like this. To me, address ranges like '1,10p' are a basic feature.)

I'm not sure I'll ever use ed for anything real (at least outside of a real emergency), but it's nice to know that I still remember enough to find my way around in it. It's also nice that the modern GNU version of ed has picked up a number of user-friendly features like the -p argument.

(The version of ed that I'm acclimatized to is usually called 'UofT ed', and features a bunch of modifications made here by a number of people, including Henry Spencer. Among other different behaviors from classical ed, it prompts with a '*' instead of nothing and has somewhat more friendly error messages.)

ActuallyUsingEd written at 20:18:45; Add Comment


Link: File crash consistency and filesystems are hard

Dan Luu's File crash consistency and filesystems are hard is about what it says it's about, with examples and interesting academic citations (that are kind of depressing) and the whole lot. If you want more reading, it has a whole bunch of links to further papers that you can go through. Most of them are apparently depressing reading, because the capsule summary is 'this stuff is hard and almost no one gets it completely right'; not user programs, not filesystems, and not the actual disk drives.

(Sometimes the 'not getting it right' bit is more accurately called 'cheating in the name of good benchmarks'.)

(I have a long standing interest in this area, as do many sysadmins. People tend to get upset if our systems lose their email during power failures and so on. Sometimes we cheat, but we like to do this with our eyes open.)

FileConsistencyHard written at 17:28:19; Add Comment

Link: Git from the inside out

Mary Rose Cook's Git from the inside out is a highly detailed and thus fascinating recounting of exactly how Git's graph structure and on disk tracking of things works as you evolve a repository. I knew many of the broad strokes just from general git knowledge but the details are illuminating and quite useful, especially the details around what exactly happens and gets recorded where during more advanced operations like merges (especially with conflicts) and pulls.

(I care about git things at this level of detail because they let me understand what's going on and what I can do about it when things don't go the way I expect them to. I'm not left poking futilely at a black box; instead I have the reassuring feeling that I can at least peek inside.)

GitInsideOut written at 17:16:12; Add Comment

(Previous 11 or go back to February 2016 at 2016/02/27)

Page tools: See As Blogdir, See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.