What free is really telling you, and some bits from /proc/meminfo

August 11, 2015

The free command is a pretty venerable Linux sysadmin tool; often it's one of the first things I reach for when I suspect that a system might be having RAM problems. Its normal output certainly looks straightforward enough, but as I discovered recently it's always good to actually look up the details of this stuff.

So let's start with the normal output:

; free -m
      total   used   free  shared  buff/cache  available
Mem:  32158  13111    683      33       18363       9727
Swap:  4094      0   4094

Total is the total physical memory. 'Used' is a calculated figure; according to the manpage, it's total minus free and buff/cache (and does not include shared). buff/cache is the sum of what 'free -w' will show separately as buffers and cache, and this is where life gets interesting.

Free's buffers is /proc/meminfo's Buffers field, which according to the fine documentation is:

Relatively temporary storage for raw disk blocks
shouldn't get tremendously large (20MB or so)

That size info is clearly what they call 'a little bit out of date', considering that our normal machines run around 400 MB of Buffers. There can be variations from this, and in particular my machine running ZFS on Linux usually has substantially more space in Buffers; right now it's almost 4 GB.

Free's cache is the sum of meminfo's Cached and Slab fields. Cached is the page cache, which is used by normal filesystems to hold pages read off disk (or in the case of NFS, off the network). Slab reports the amount of memory used by the kernel's slab allocator. Calling this cache is potentially misleading, since the slab allocator is widely used all through the kernel for many, many things as you can see by looking at /proc/slabinfo or running slabtop, in both cases as root (note that slab merging can make the slab names somewhat misleading). However it's true that some of the kernel's slab memory usage is for 'caches' (broadly construed).

Free's available is meminfo's MemAvailable, which is a kernel estimate based on a bunch of internal heuristics that I'm not going to try to write up. These heuristics can be wrong for various reasons, including kernel subsystems that can shrink their slab usage under memory pressure but don't put the right magic markers on their slabs so that some of their slab usage shows up in MemAvailable.

Nothing in free's output tells you how much memory the kernel is using in total. The kernel uses more memory than appears in buffers and cache, and free does not attempt to reverse engineer a kernel usage figure based on finding out how much memory appears to be in use by user programs. This is fair enough, given that free is primarily concerned with telling you how much memory is or might be freeable (it's in its name, after all). Still, it's something to bear in mind; if you really want to get a picture of kernel memory usage, you want another tool.

(I don't know what tool you want; I haven't looked yet. I suspect that sufficient information is exposed in /proc/meminfo to work this out with reasonable accuracy.)

PS: If it was up to me, I would redefine free's cached to be meminfo's Cached plus SReclaimable. The latter is the amount of space used by slabs that have explicitly marked themselves as reclaimable under memory pressure, which is hopefully more or less the set of slabs that are actually being used as caches instead of for more sticky things.


Comments on this page:

By Alan at 2015-08-11 03:40:33:

You may also have noticed free changed recently. Compare Debian 8:

 $ free
              total       used       free     shared    buffers     cached
 Mem:        513664     494704      18960      18496      68824     336472
 -/+ buffers/cache:      89408     424256
 Swap:      1048572      38212    1010360

Most of the time, I use free solely as a zeroth level diagnostic: is something grossly wrong with memory usage? And by that, mostly I mean "does this machine have the physical amount of RAM I expect, and is more than a token amount of swap being used?"

If I login to a server and see an underwhelming physical RAM, either something went wrong in hardware or I am looking at a machine which is substantially older than the rest of our fleet. If I see significant swap usage, I then have to look at what is using memory - is this a case of process(es) run amok, or a simple case of overloading the box?

By Paulo Almeida at 2015-08-11 11:10:22:

Regarding Alan's comment, this is not a recent change (depending on how "recent" is defined, naturally). I have been seeing that output for a long time, and it's present at least in Debian 6, which is the oldest I still have installed (the man page says it's from 2009).

By the way, it was good to read the man page for this, because I just realized there's a "free --human" option that I'll use from now on. This option exists in Debian 8 but not 6.

By Alan at 2015-08-11 13:21:52:

I should have been clearer: Debian has the older version :).

Written on 11 August 2015.
« My irritation with 'systemctl status'
What common older versions of free are telling you »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Aug 11 01:56:23 2015
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.