Finding out what 32-bit x86 Linux programs your users are running
Canonical has recently created a certain amount of uncertainty over how many 32-bit x86 programs are still going to run on Ubuntu 20.04 LTS and future versions. Statically linked 32-bit programs seem likely to keep working, if you have any of those, but Canonical is making noises about omitting some or many of the 32-bit shared libraries that dynamically linked 32-bit programs need to run (see my entry on what 32-bit programs need on a 64-bit system). This is quite relevant for us, as we definitely have a certain number of users with a certain amount of old 32-bit programs.
(Canonical's latest statement is not entirely clear, and anyway it only applies to 19.10 and 20.04 LTS, not versions past them.)
Given all of this, it would be nice to find out (in advance) what 32-bit programs our users are still running. There are probably a number of different ways to do this, but the approach I've explored is using the Linux kernel's audit system. The audit framework can log both the use of 32-bit system calls and the use of various paths, so there are several specific approaches you can take. I will start with the one that doesn't work.
The obvious thing to attempt is to record 32-bit
works in one sense, in that it does what it says on the can, but
it doesn't in another sense; it records when a 32-bit program uses
execve(), not when an
execve() results in a 32-bit program
being run. As far as I can see, there is no direct way of doing the
latter; instead we have to do it indirectly.
If you're content to log a great deal of audit records, you can
record many or basically all use of 32-bit system calls. This will
definitely catch 32-bit programs, but it's noisy; you'll log a lot
of records for each program. To do better we need to find a system
call, perhaps with a specific set of arguments, that is only used
relatively infrequently in 32-bit programs. Fortunately there is
one, at least for programs written in C or using the standard C
library entry point, and it is '
brk(NULL)', which is run very
early on in the program's life to find the break address. In audit syntax, this is:
-a always,exit -F arch=b32 -F a0=0 -S brk -k 32bit-program
If it matters to you, this audit rule will not catch statically linked 32-bit Go programs (although it will catch statically linked 32-bit C programs). Of course as statically linked programs, they don't care if Canonical drops 32-bit shared libraries.
If we specifically want to catch 32-bit dynamically linked programs,
we can use an audit path watch on
there is a complication;
ld-linux.so.2 is a symlink, so we must
watch the specific thing it points to, and that depends on the
version of GNU libc you have installed and on the system you're on.
So, first you use '
readlink -f /lib/ld-linux.so.2' to find the
expanded path, then put the result into an audit rule like so:
-w /lib/i386-linux-gnu/ld-2.27.so -p x -k 32bit-program
(This is for Ubuntu 18.04, with GNU libc 2.27.)
You can also set an audit path watch on the main C shared library,
libc.so.6, or for that matter on any shared library of interest
or that you think is especially in danger of not being available
in future Ubuntu versions. Just remember to always expand it out
to the real path or you'll get audit rules that mysteriously don't
work (ask me how I know this). If you want, you can be specific
openat() system calls for the path, since GNU libc
uses that when mmap()'ing shared libraries.
If you want to get audit records from statically linked non-C programs,
you'll need to use
strace to investigate what system calls they
make. A cross-built statically linked 32-bit Go program appears to
sched_getaffinity early on, for
example. Your mileage will vary with other languages and other C