== What common older versions of _free_ are telling you

[[Yesterday I wrote about what _free_ was really telling you
FreeAndMeminfo]], using the output (and manpage) from the version
of _free_ on Fedora 22. As Alan noted in a comment on my entry,
there are actually two versions of _free_, with somewhat different
output. The older version reports:

.pn prewrap on
 ; free -m
              total   used   free  shared  buffers  cached
 Mem:         16018  15760    258       3       72   12924
 -/+ buffers/cache:   2763  13254
 Swap:         3811     19   3792

This version is apparently on Debian machines and is definitely on
Ubuntu up to 14.04 LTS and RHEL/CentOS up to RHEL 7, which covers
most of the machines out there. The change in output turns out to
have been introduced in [[procps-ng https://gitlab.com/procps-ng/procps]]
3.3.10, which was released around September of 2014. Very few
distributions seem to have picked up the new version, although it
is in Debian testing (and Fedora, obviously).

(Although Debian and Ubuntu call the package 'procps', they're
actually using procps-ng. The history here is tangled.)

The useful thing to know here is that in this older version of
_free_, what is reported in _cached_ is *only* the _/proc/meminfo_
_Cached_ field. The _Slab_ memory usage is not reported anywhere,
which matters because a certain amount of kernel caches are actually
in slabs, not in the page cache.

So, you might ask, how much kernel memory is typically used in
slabs? The answer appears to be 'generally not much'. On our fleet
of machines slab usage seems to typically run in the 150 MB to 300
MB range; several machines have several hundred MB, and a few
exceptions have around 1 GB to 1.5 GB of slab usage. My office
workstation is a drastic outlier, but that's because [[ZFS on Linux
http://zfsonlinux.org/]] currently puts much of the ZFS ARC into
the slab allocation system instead of having it as part of the page
cache.

(On our big-slab machines, '_slabtop -s c_' says that most of the
slab usage seems to be eaten up by dentry and inode slabs, either
for local filesystems or for NFS depending on the machine.)

(This is yet another case where writing entries here has turned out
to be educational.)