Wandering Thoughts archives


Making two Unix permissions mistakes in one

I tweeted:

Today's state of work-brain:
mkdir /tmp/fred
umask 077 /tmp/fred

Immediately after these two commands, I hit cursor-up to change the 'umask' to 'chmod', so that I then ran 'chmod 077 /tmp/fred'. Fortunately I was doing this as a regular user, so my next action exposed my error.

This whole sequence of commands is a set of mistakes jumbled together in a very Unix way. My goal was to create a new /tmp/fred directory that was only accessible to me. My second command is not just wrong because I wanted chmod instead of umask (I should have run umask before the mkdir, not after), but because I had the wrong set of permissions for chmod. It was as if my brain wanted Unix to apply a 'umask 077' to the creation of /tmp/fred after the fact. Since the numeric permissions you give to umask are the inverse of the permissions you give to chmod (you tell umask what you don't want instead of what you do), my change of umask to chmod then left /tmp/fred with completely wrong permissions; instead of being only accessible to me, it was fully accessible to everyone except me.

(Had I been doing this as root, I would then have been able to cd into the directory, put files in it, access files in it, and so on, and might not have noticed that the permissions were reversed from what I actually wanted.)

The traditional Unix umask itself is a very Unix command (well, shell built-in), in that it more or less directly calls umask(). This allows a very simple implementation, which was a priority in early Unixes like V7. A more sensible implementation would be that you specify effectively the maximum permissions that you want (for example, that things can be '755') and then umask would invert this to get the value it uses for umask(). But early Unixes took the direct approach, counting on people to remember the inversion and perform it in their heads.

In the process of writing this entry I learned that POSIX umask supports symbolic modes, and that they work this way. You get and set umask modes like 'u=rwx,g=rx,o=rx' (aka '022', the traditional friendly Unix umask), and they're the same permissions as you would use with chmod. I believe that this symbolic mode is supported by any modern Bourne compatible shell (including zsh), but it isn't necessarily supported by non-Bourne shells such as tcsh or rc (which is my shell).

unix/PermissionsTwoMistakes written at 23:53:11; Add Comment

Some ways to get (or not get) information about system memory ranges on Linux

I recently learned about lsmem, which is described as "list[ing] the ranges of available memory [...]". The source I learned it from was curious why lsmem on a modern 64-bit machine didn't list all of the low 4 GB as a single block (they were exploring kernel memory zones, where the low 4 GB of RAM are still a special 'DMA32' zone). To start with, I'll show typical lsmem default output from a machine with 32 GB of RAM:

; lsmem
RANGE                                  SIZE  STATE REMOVABLE  BLOCK
0x0000000000000000-0x00000000dfffffff  3.5G online       yes   0-27
0x0000000100000000-0x000000081fffffff 28.5G online       yes 32-259

Memory block size:       128M
Total online memory:      32G
Total offline memory:      0B

Lsmem is reporting information from /sys/devices/system/memory (see also memory-hotplug.txt). Both the sysfs hierarchy and lsmem itself apparently come originally from the IBM S390x architecture. Today this sysfs hierarchy apparently only exists for memory hotplug, and there are some signs that kernel developers aren't fond of it.

(Update: I'm wrong about where the sysfs memory hierarchy comes from; see this tweet from Dave Hansen.)

On the machines I've looked at, the hole reported by lsmem is authentic, in that /sys/devices/system/memory also doesn't have any nodes for that range (on the machine above, for blocks 28, 29, 30, and 31). The specific gap varies from machine to machine. However, all of the information from lsmem may well be a simplification of a more complex reality.

The kernel also exposes physical memory range information through /proc in /proc/iomem (on modern kernels you'll probably have to read this as root to get real address ranges). This has a much more complicated view of actual RAM, one with many more holes than what lsmem and /sys/devices/system/memory show. This is especially the case in the low 4G of memory, where for example the system above reports a whole series of chunks of reserved memory, PCI bus address space, ACPI tables and storage, and more. The high memory range is simpler, but still not quite the same:

100000000-81f37ffff : System RAM
81f380000-81fffffff : RAM buffer

The information from /proc/iomem has a lot of information about PCI(e) windows and other things, so you may want to narrow down what you look at. On the system above, /proc/iomem has 107 lines but only nine of them are for 'System RAM', and all but one of them are in the physical memory address range that lsmem lumps into the 'low' 3.5 GB:

00001000-0009d3ff : System RAM
00100000-09e0ffff : System RAM
0a000000-0a1fffff : System RAM
0a20b000-0affffff : System RAM
0b020000-d17bafff : System RAM
d17da000-da66ffff : System RAM
da7e5000-da8eefff : System RAM
dbac7000-ddffffff : System RAM

(I don't have the energy to work out how much actual RAM this represents.)

Another view of physical memory range information is the kernel's report of the BIOS 'e820' memory map, printed during boot. On the system above, this says that the top of memory is actually 0x81f37ffff:

BIOS-e820: [mem 0x0000000100000000-0x000000081f37ffff] usable

I don't know if the Linux kernel exposes this information in /sys. You can also find various other things about physical memory ranges in the kernel's boot messages, but I don't know enough to analyze them.

What's clear is that in general, a modern x86 machine's physical memory ranges are quite complicated. There are historical bits and pieces, ACPI and other data that is in RAM but must be preserved, PCI(e) windows, and other things.

(I assume that there is low level chipset magic to direct reads and writes for RAM to the appropriate bits of RAM, including remapping parts of the DIMMs around so that they can be more or less fully used.)

linux/SystemMemoryRangeInfo written at 01:00:13; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.