Wandering Thoughts archives

2017-03-30

What top's SHR field means in quite modern Linux kernels

A number of years ago I wrote an entry on what you can find out about the memory usage of your Linux programs, and in it I cast cold water on what top reports as SHR on the grounds that it was only vaguely documented. There are two pieces of good news. The first is that I can now tell you what the field means and where it comes from, and the second is that it is actually quite well documented in the latest top manpage. That top manpage is well worth your attention, because current versions have added a really useful section called Linux Memory Types, which covers what you might think it does in a nice clear way.

(Unfortunately your distribution probably doesn't have an up to date top manpage. Top comes as part of procps-ng, and this section of the manpage seems to only have appeared in procps-ng 3.3.12, in a commit made in 2016. Even Fedora 25 only has procps-ng 3.3.10, which is too old to have the nice version of the manpage.)

To summarize the manpage, SHR is a subset of top's RES, which is the process's resident set size, and counts resident pages of memory that might be shared with other processes in a number of ways that include shared POSIX shared memory and shareable mmap()s of files. More specifically, it is the shared field from /proc/PID/statm, which is described in the kernel documentation as:

number of pages that are shared (i.e. backed by a file, same as RssFile+RssShmem in status)

The same documentation describes these fields in /proc/PID/status as:

RssFile
size of resident file mappings
RssShmem
size of resident shmem memory (includes SysV shm, mapping of tmpfs and shared anonymous mappings)

This is two out of the three components of a process's RSS, the third being RssAnon ('size of resident anonymous memory', unsurprisingly). According to that top manpage, this three-way split of RSS apparently came into existence around kernel 4.5.

In the actual kernel code, these three measures count MM_FILEPAGES, MM_SHMEMPAGES, and MM_ANONPAGES pages respectively, and this is about where my desire to climb down this rabbit hole runs out. I think we might as well take the kernel documentation and top's current manpage at their word and assume that they mean what they say (for kernel 4.5 onwards).

At this point you might be curious about what happens in a Linux kernel before 4.5, especially since Ubuntu 16.04 has a 4.4.0 based kernel. As far as I can decipher from the Ubuntu 16.04 kernel source, the only real change is that process RSS had only two components, 'resident file mappings' and 'resident anonymous mappings', and I think that pages that would now be counted in RssShmem wind up in RssFile instead. If this is correct, then the actual value for statm's shared and thus top's SHR is going to be the same between a 4.4 kernel and a later one; it's just that if you look at /proc/PID/status from 4.5 onwards, you can get a bit more information.

(Although 4.4 and earlier don't give you an explicit RssAnon field, you can deduce its value by subtracting the statm 'shared' value from the total 'resident' value. You have to read statm to get this, since /proc/PID/status only gives you the combined VmRSS value.)

linux/TopSHRFieldMeaning written at 00:01:12;


Page tools: See As Normal.
Search:
Login: Password:

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.