Wandering Thoughts archives

Some things about changing from old X fonts to modern TrueType fonts

Now that I've transitioned to using modern fonts in xterm, I've decided to move into the 00s by changing to a UTF-8 based text encoding (I've considered this some time, but only once I had more or less Unicode-enabled fonts did it become actively tempting). Making this shift provoked a small pile of complaints from FVWM about my bitmap fonts not having all of the font charsets that a true Unicode environment wants, which caused me to start switching various other fonts to modern TrueType fonts.

One of my challenges in doing this was matching the size of my new fonts to the size of my old fonts, since I didn't want to significantly change the size of things in my desktop environment; shuffling fonts is bad enough without shuffling layouts as well. Fortunately there turns out to be a relatively easy trick for this that mostly works.

There are two ways to specify the sizes of old-style X fonts; you can use either actual pixel size or a point size (or both at once, if you're so inclined). If you only have the point size, you need to determine the pixel size. The easy way to do this is to fire up xfontsel like so:

xfontsel -pattern "-adobe-helvetica-bold-r-normal--*-120-100-100-*-*-*-*"

Now find the pxlsz dropdown, and it should normally have exactly one pixel size (in this case, 17 pixels).

Once you have the pixel size you can simply directly specify this in an XFT font specification such as 'Sans:Bold:pixelsize=17'. This will give you a font that is either exactly or almost exactly the same vertical size as your previous old X font. I don't know if there's a magic XFT way to work out the correct (point) size that corresponds to a specific pixel size; so far I haven't worried about it.

(The font rendition may not be the same, of course; in my case I found that I needed 'Condensed Bold' instead of plain 'Bold' in order to match reasonably well. You're going to have to experiment.)

The other thing I found out after some experimentation is that modern TrueType fonts are not as good at small pixel sizes as the old X bitmap fonts. After playing around with it I've stuck with my old X fonts for small sized things (an example is the places where I'm using the classic 8x13), complaints from FVWM and all.

(It's possible that someday we'll have modern TrueType fonts that are very well hinted for such small sizes, but they don't seem to be there today. It's not glaringly bad by comparison, but my FvwmIconMan definitely felt less readable and clear when I was trying out 'Mono:pixelsize=13'.)

Sidebar: Xt and bad font rendering in UTF-8

One of the things that happens in some Xt-based applications, most visibly xclock, is that if you use a UTF-8 based text encoding things suddenly insist on doubling the vertical space they use for rendering each line of text. I don't know why they do this, but so far my only solution is to turn off LC_CTYPE when I start those applications so that they revert to the default C locale.

Understanding FVWM States, with better FVWM code for selecting terminal windows

Every so often I make an epic mistake, where I drastically misunderstand something. A discussion with one of the people behind FVWM in response to yesterday's entry on selecting terminal windows from the keyboard revealed that I had made one of them about an FVWM feature called (window) 'States'; a correct understanding of the feature leads to a better version of the FVWM code from my last entry.

What FVWM States are is effectively user-controlled tags for windows, given numbers from 0 through 31 instead of (say) text labels. Crucially, States are completely independent from each other; a given window can be in multiple States at once, and individual States are manipulated independently. Windows default to having all States turned off but can have this changed by FVWM Style commands or on the fly. States can be used for at least two separate things. The first is to keep track of actual window states, such as whether or not a window has ever been iconified; the second is to aggregate windows together into some sort of abstract category such as 'all terminal windows'.

(My mistake was thinking that a window could only be in one State at a time, which meant that using States to identify terminal windows would preclude using States for anything else involving them, more or less. This is wrong.)

The second sort of use of States is the interesting one for what I was doing in yesterday's entry, so here's revised FVWM code with commentary. First, we mark all windows we consider 'terminal windows' with a specific State through a Style command. I've picked State 2 here, but the choice is arbitrary (provided that you aren't already using the state number).

Style "XTerm"            State 2
Style "9term"            State 2
Style "Gnome-terminal"   State 2

Any time your FVWM configuration wants to specify 'this applies to terminal windows' it can now simply use 'State 2' as the matching condition. Specifically we can do this in my ToWindow function:

AddToFunc ToWindow
+ I   Current (State 2, "$0") Break
+ I   Next (State 2, "$0") ToWindow2

AddToFunc ToWindow2
+ I   Iconify False
+ I   Focus
+ I   Raise
+ I   WarpToWindow 80 20

Using a State to identify terminal windows has the great advantage that if you start using another form of terminal window, such as Konsole, you don't have to go through your FVWM configuration to add additional '|Konsole' matching conditions to everything; you just add a single Style declaration to identify Konsole as a terminal.

Style "Konsole"          State 2

(If you actually want all terminal windows to have the same window manager decorations and so on, you can do even better than this. FVWM allows for abstract styles, so you can create a 'terminal' style that decorates the window appropriately and also includes the 'State 2' setting, then simply say 'Style "Konsole" UseStyle terminal'.)

(I'd like to thank Thomas Adam for taking the time to patiently get enlightenment into my head.)

How I select (terminal) windows from the keyboard in FVWM

As part of reorganizing my work desktop to deal with big screens, I decided that I wanted a way to select terminal windows by name from the keyboard, with some form of autocompletion for speed. The goal was to give me a faster way to get a particular terminal window than sweeping my mouse all the way to my taskbar for terminals and finding the specific window I wanted. FVWM doesn't have any native features for this, so I had to bolt something together for this out of pieces.

(To make life simple I decided to not do anything clever about several windows having the same name. If that happened and I picked the name, the system could pick whichever specific window it wanted.)

The most difficult bit is selecting among alternatives from the keyboard with some form of autocompletion. Fortunately I already had a general program that could do this: dmenu, which I was already using as the core of a general 'do things' launcher. Dmenu will read a list of things from standard input, put up a menu/text entry area, let you type to it, and then print out the selection to standard output. All I needed was a way of generating the list of terminal window names and then of making FVWM activate the chosen window.

With the help of the FVWM mailing list I managed to put together both parts of this. Here is what I have. First, the driver script (slightly simplified; the real version also matches gnome-terminal):

#!/bin/sh

# Generate a list of unique names of
# terminal windows.
genwins() {
  xwininfo -root -tree |
  egrep '': \([^)]* "(XTerm|9term)"\)  [0-9]' |
  awk '{print $2}' |
  sed -e 's/^"//' -e 's/":$//' |
  sort -u
}
# run dmenu to get the answer.
win=$(genwins | dmenu -b -p win -P -t) || exit 1
# pass the selected window to FVWM
echo "Function ToWindow \"$win\"" |
  FvwmCommand -c

(The -P and -t arguments to dmenu are from a personal patch that I did. -t is especially useful because it makes dmenu do shell-like partial autocompletion. The real script also specifies the dmenu colours and font to use. There may be a more elegant command than xwininfo to use for getting the window names, but xwininfo has the virtue of being ready to hand.)

In FVWM, I've defined the ToWindow function as follows (the real one also selects gnome-terminal windows):

 AddToFunc ToWindow
 + I   Current ("XTerm|9term", "$0") Break
 + I   Next ("XTerm|9term", "$0") ToWindow2

 AddToFunc ToWindow2
 + I   Iconify False
 + I   Focus
 + I   Raise
 + I   WarpToWindow 80 20

Update: there is a better way to write ToWindow using FVWM States; I've written it up in FvwmStatesUnderstood.

This does nothing if either there is no such terminal window or the current window is a terminal window with the right name. Otherwise, the named terminal window is deiconified, given the focus (which also switches me to its virtual screen if it's not on the current virtual screen), raised to be on top of everything else, and then the mouse pointer is moved into it. I don't normally like having the mouse pointer moved around on me, but it's necessary here in order to make sure that this window keeps the focus; popping up a window and then immediately having it lose focus was more irritating than having my mouse pointer teleported.

(The odd structure of this as two functions is necessary for internal FVWM reasons. Basically it's the easiest way to insure that nothing happens if there's no such window.)

(I also had to add FvwmCommandS (the FvwmCommands server module) to the set of FVWM modules that my configuration runs, since I hadn't been using it before this.)

PS: This code omits '|Gnome-terminal' in three places purely to shorten the length of the unbreakable lines in this blog entry.

Understanding Resident Set Size and the RSS problem on modern Unixes

On a modern Unix system with all sorts of memory sharing between processes, Resident Set Size is a hard thing to explain; I resorted to a very technical description in my entry on Linux memory stats. To actually understand RSS, let's back up and imagine a hypothetical old system that has no memory sharing between processes at all; each page of RAM is either free or in use by exactly one process.

(We'll ignore the RAM the operating system itself uses. In old Unixes, this was an acceptable simplification; memory was statically divided between memory used by the OS and memory used by user programs.)

In this system, processes acquire new pages of RAM by trying to access them and then either having them allocated or having them paged (back) in from disk. Meanwhile, the kernel is running around trying to free up memory, generally using some approximation of finding the least recently used page of RAM. How aggressively the operating system tries to reclaim pages depends on how much free memory it has; the less free memory, the faster the OS tries to grab pages back. In this environment, the resident set size of a process is how many pages of RAM it has. If the system is not thrashing, ie if there's enough memory to go around, a process's RSS is how much RAM it actually needs in order to work at its current pace.

(All of this is standard material from an operating system course.)

The problem of RSS on modern Unix systems is how to adopt this model to an environment where processes share significant amounts of memory with each other. In the face of a lot of sharing, what does it mean for a process to have a resident set size and how do you find the right pages to free up?

There are at least two approaches the kernel can take to reclaiming pages, which we can call the 'physical' and 'process' approaches. In the physical approach the kernel continues to scan over physical RAM to identify candidate pages to be freed up; when it finds one, it takes it away from all of the processes using it at once (this is the 'global' removal of my earlier entry). In the process approach the kernel scans each process more or less independently, finding candidate pages and removing them only from the process (a 'local' removal); only once a candidate page has been removed from all processes using it is it actually freed up.

(Scanning each 'process' is a simplification. Really the kernel scans each separate set of page tables; there are situations where multiple processes share a single set of page tables.)

The problem with the process approach is that the kernel can spend a great deal of time removing pages from processes when the pages will never actually be reclaimed for real. Imagine two processes with a shared memory area; one process uses it actively and one process only uses it slowly. The kernel can spend all the time it wants removing pages of the shared area from the less active process without ever actually getting any RAM back, because the active process is keeping all of those pages in RAM anyways.

So, why doesn't everyone use the physical approach? My understanding is that the problem with the physical approach is that it is often not necessarily a good fit for how the hardware manages virtual memory activity information. Per my earlier entry, every process mapping a shared page of RAM can have a different page table entry for it. To find out if the page of RAM has been accessed recently you may have to find and look at all of those PTEs (with locking), and do so for every page of physical RAM you look at.

My impression is that most current Unixes normally use per-process scanning, perhaps falling back on physical scanning if memory pressure gets sufficiently severe.

(I suspect and hope that virtual memory management in the face of shared pages have been studied academically, just as the older and simpler model of virtual memory has been, but I'm out of contact with OS research.)