Finding out what 32-bit x86 Linux programs your users are running

July 3, 2019

Canonical has recently created a certain amount of uncertainty over how many 32-bit x86 programs are still going to run on Ubuntu 20.04 LTS and future versions. Statically linked 32-bit programs seem likely to keep working, if you have any of those, but Canonical is making noises about omitting some or many of the 32-bit shared libraries that dynamically linked 32-bit programs need to run (see my entry on what 32-bit programs need on a 64-bit system). This is quite relevant for us, as we definitely have a certain number of users with a certain amount of old 32-bit programs.

(Canonical's latest statement is not entirely clear, and anyway it only applies to 19.10 and 20.04 LTS, not versions past them.)

Given all of this, it would be nice to find out (in advance) what 32-bit programs our users are still running. There are probably a number of different ways to do this, but the approach I've explored is using the Linux kernel's audit system. The audit framework can log both the use of 32-bit system calls and the use of various paths, so there are several specific approaches you can take. I will start with the one that doesn't work.

The obvious thing to attempt is to record 32-bit execve()s. This works in one sense, in that it does what it says on the can, but it doesn't in another sense; it records when a 32-bit program uses an execve(), not when an execve() results in a 32-bit program being run. As far as I can see, there is no direct way of doing the latter; instead we have to do it indirectly.

If you're content to log a great deal of audit records, you can record many or basically all use of 32-bit system calls. This will definitely catch 32-bit programs, but it's noisy; you'll log a lot of records for each program. To do better we need to find a system call, perhaps with a specific set of arguments, that is only used relatively infrequently in 32-bit programs. Fortunately there is one, at least for programs written in C or using the standard C library entry point, and it is 'brk(NULL)', which is run very early on in the program's life to find the break address. In audit syntax, this is:

-a always,exit -F arch=b32 -F a0=0 -S brk -k 32bit-program

If it matters to you, this audit rule will not catch statically linked 32-bit Go programs (although it will catch statically linked 32-bit C programs). Of course as statically linked programs, they don't care if Canonical drops 32-bit shared libraries.

If we specifically want to catch 32-bit dynamically linked programs, we can use an audit path watch on /lib/ However, there is a complication; is a symlink, so we must watch the specific thing it points to, and that depends on the version of GNU libc you have installed and on the system you're on. So, first you use 'readlink -f /lib/' to find the expanded path, then put the result into an audit rule like so:

-w /lib/i386-linux-gnu/ -p x -k 32bit-program

(This is for Ubuntu 18.04, with GNU libc 2.27.)

You can also set an audit path watch on the main C shared library,, or for that matter on any shared library of interest or that you think is especially in danger of not being available in future Ubuntu versions. Just remember to always expand it out to the real path or you'll get audit rules that mysteriously don't work (ask me how I know this). If you want, you can be specific by watching openat() system calls for the path, since GNU libc uses that when mmap()'ing shared libraries.

If you want to get audit records from statically linked non-C programs, you'll need to use strace to investigate what system calls they make. A cross-built statically linked 32-bit Go program appears to call set_thread_area and sched_getaffinity early on, for example. Your mileage will vary with other languages and other C libraries.

Written on 03 July 2019.
« The power of option types is in what they do to the rest of the language
Converting a variable to a single-element slice in Go via unsafe »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Jul 3 00:53:42 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.