Wandering Thoughts archives

The perils of having an ancient $HOME (a yak shaving story)

I build Firefox from source for obscure reasons, and for equally obscure reasons I do this from the Firefox development repo instead of, say, the source for the current release. Mozilla has recently been moving towards making Rust mandatory for building Firefox (see especially the comments). If you build the Mercurial tip from source this is currently optional, but I can see the writing on the wall so I decided to get a copy of Rust and turn this option on. If nothing else, I could file bugs for any problems I ran into.

Getting a copy of Rust went reasonably well (although Rust compiles itself painfully slowly) and I could now build the Rust component without issues. Well, mostly without issues. During part of building Firefox, rustc (the Rust compiler) would print out:

[...]
librul.a
note: link against the following native artifacts when linking against this static library
note: the order and any duplication can be significant on some platforms, and so may need to be preserved
note: library: dl
note: library: pthread
note: library: gcc_s note: library: c
note: library: m
note: library: rt
note: library: util
libxul_s.a.desc
libxul.so
Exporting extension to source/test/addons/author-email.xpi.
[...]

Everything from the first 'note:' onwards was in bold, including later messages (such as 'Exporting ...') that were produced by other portions of the build process. Clearly rustc was somehow forgetting to tell the terminal to turn off boldface, in much the same way that people writing HTML sometimes forget the '</b>' in their markup and turn the rest of the page bold.

This only happened in xterm; in gnome-terminal and other things, rustc turned off bold without problems. This didn't surprise me, because almost no one still uses and thus tests with xterm these days. Clearly there was some obscure detail about escape sequence handling that xterm was doing differently from gnome-terminal and other modern terminal emulators, and this was tripping up rustc and causing it to fail to close off boldface.

(After capturing output with script, I determined that rustc was both turning on bold and trying to turn it off again with the same sequence, 'ESC [ 1 m'. Oh, I said to myself, this has clearly turned into a toggle in modern implementations, but xterm has stuck grimly to the old ways and is making you do it properly. This is the unwonted, hubristic arrogance of the old Unix hand speaking,.)

Since life is short and my patience is limited, I dealt with this simply; I wrote a cover script for rustc that manually turned off bold (and in fact all special terminal attributes) afterwards. It was roughly:

#!/bin/sh
rustc.real "$@"
st=$?; tput sgr0; exit $st

This worked fine. I could still build Firefox with the Rust bits enabled, and my xterm no longer became a sea of boldface afterwards.

Today I ran across an announcement of git-series, an interesting looking git tool to track the evolution of a series of patches over time as they get rebased and so on. Since this is roughly how I use git to carry my own patches on top of upstream repos, I decided I was interested enough to take a look. Git-series is written in Rust, which is fine because I already had that set up, but it also requires Rust's package manager Cargo. Cargo doesn't come with Rust's source distribution; you have to get it and build it separately. Cargo is of course built in Rust. So I cloned the Cargo repo and started building.

It didn't go well. In fact it blew up spectacularly and mysteriously right at the start. Alex Crichton of the Rust project took a look at my strace output and reported helpfully that something seemed to be adding a stray ESC byte to rustc's output stream when the build process ran it and tried to parse the output.

Oh. Well. That would be the tput from my cover script, wouldn't it. I was running tput even when standard output wasn't a terminal and this was mucking things up for anything that tried to consume rustc output. That fixed my issue with Cargo, but now I wanted to get this whole 'doesn't turn off boldface right' issue in Rust fixed, so I started digging to at least characterize things so I could file a bug report.

In the process of poking and prodding at this, a little bit of my strace output started nagging at me; there had been references in it to /u/cks/.terminfo. I started to wonder if I had something outdated in my personal environment that was confusing Rust's terminfo support library into generating the wrong output. Since I couldn't remember anything important I had set up there, I renamed it to .terminfo-hold and re-tested.

Magically, everything now worked fine. Rustc was happy with life.

What did I have in .terminfo? Well, here:

; ls -l /u/cks/.terminfo-hold/x/xterm
-rw-r--r--. 2 cks cks 1353 Feb 10  1998 .terminfo-hold/x/xterm

Here in 2016, I have no idea why I needed a personal xterm terminfo file in 1998, or what's in it. But it's from 1998, so I'm pretty confident it's out of date. If it was doing something that I'm going to miss, I should probably recreate it from a modern xterm terminfo entry.

(This also explains why gnome-terminal had no problems; it normally has a $TERM value of xterm-256color, and I didn't have a custom terminfo file for that.)

My $HOME is really old by now, and as a result it has all sorts of ancient things lingering on in its dotfile depths. Some of them are from stuff I configured a long time ago and have forgotten since, and some of them are just the random traces from ancient programs that I haven't used for years. Clearly this has its hazards.

(Despite this experience I have no more desire to start over from scratch than I did before. My environment works and by now I'm very accustomed to it.)

Sudo and changes in security expectations (and user behaviors)

Sudo has a well known, even famous default behavior; if you try to use sudo and you don't have sudo privileges, it sends an email alert off to the sysadmins (sometimes these are useful). In my view, this is the sign of a fundamental assumption in sudo's security model, namely that it's only going to be used by authorized people or by malicious parties. If you're not a sysadmin or an operator or so on, you know that you have no business running sudo, so you don't. Given the assumption that unauthorized people don't innocently run sudo, it made sense to send alert email about it by default.

Once upon a time that security model was perfectly sensible, back in the days when Unix machines were big and uncommon and theoretically always run by experienced professionals. Oh, sure, maybe the odd sysadmin or operator would accidentally run sudo on the wrong machine, but you could expect that ordinary people would never touch it. Today, however, those days are over. Unix machines are small and pervasive, there are tons of people who have some involvement in sysadmin things on one, and sudo has been extremely successful. The natural result is that there are a lot of people out there who are following canned howto instructions without really thinking about them, and these instructions say to use sudo to get things done.

(Sometimes the use of sudo is embedded into an installation script or the like. The Let's Encrypt standard certbot-auto script works this way, for instance; it blithely uses sudo to do all sorts of things to your system without particularly warning you, asking for permission, or the like.)

In other words, the security model that there's basically no innocent unauthorized use of sudo is now incorrect, at least on multi-user Unix systems. There are plenty of such innocent attempts, and in some environments (such as ours) they're the dominant ones. Should this cause sudo's defaults to change? That I don't know, but the pragmatic answer is that in the grand Unix tradition, leaving the defaults unchanged is easier.

(There remain Unix environments where there shouldn't be any such unauthorized uses, of course. Arguably multi-user Unix environments are less common now than such systems, where you very much do want to get emailed if, eg, the web server UID suddenly tries to run sudo.)

cal's unfortunate problem with argument handling

Every so often I want to see a calendar just to know things like what day of the week a future date will be (or vice versa). As an old Unix person, my tool for this is cal. Cal is generally a useful program, but it has one unfortunate usage quirk that arguably shows a general issue with Unix style argument handling.

By default, cal just shows you the current month. Suppose that you are using cal at the end of June, and you decide that you want to see July's calendar. So you absently do the obvious thing and run 'cal 7' (because cal loves its months in decimal form). This does not do what you want; instead of seeing the month calendar for July of this year, you see the nominal full year calendar for AD 7. To see July, you need to do something like 'cal 7 2016' or 'cal -m 7'.

On the one hand, this is regrettably user hostile. 'cal N' for N in the range of 1 to 12 is far more likely to be someone wanting to see the given month for the current year than it is to be someone who wants to see the year calendar for AD N. On the other hand, it's hard to get out of this without resorting to ugly heuristics. It's probably equally common to want a full year calendar from cal as it is to want a different month's calendar, and both of these operations would like to lay claim to the single argument 'cal N' invocation because that's the most convenient way to do it.

If we were creating cal from scratch, one reasonably decent option would be to declare that all uses of cal without switches to explicitly tell it what you wanted were subject to heuristics. Then cal would have a license to make 'cal 7' mean July of this year instead of AD 7, and maybe 'cal 78' mean 'cal 1978' (cf the note in the V7 cal manpage). If you really wanted AD 7's year calendar, you'd give cal a switch to disambiguate the situation; in the mean time, you'd have no grounds for complaint. But however nice it might be, this would probably strike people as non-Unixy. Unix commands traditionally have predictable argument handling, even if it's not friendly, because that's what Unix considers more important (and also easier, if we're being honest).

In a related issue, I have now actually read the manpages for modern versions of cal (FreeBSD and Linux use different implementations) and boy has it grown a lot of options by now (options that will probably make my life easier if I can remember them and remember to use them). Reassuringly, the OmniOS version of cal still takes no switches; it's retained the V7 'cal [[month] year]' usage over all of these years.