The ARC now seems to work right in ZFS on Linux

August 8, 2015

One of the long standing with ZFS on Linux has been its integration with the general Linux kernel memory allocation system. In particular, back in December I wrote about my problem of memory competition between the ZFS ARC and the general Linux page cache, where my much smaller ext4 root filesystem wound up with a lot of data cached and the ARC was much too small. The good news is that a lot can change in seven months.

As of some recent updates to the development version of ZFS on Linux, this problem has vanished on my machine. I now consistently see the ARC being a decent size and holding a significant amount of data, and the ext4 page cache seems to be modest (you have to sort of observe the ext4 page cache by implication, based on the difference between ARC memory usage and what 'free' reports about overall kernel buffer and cache usage). In short, the ZFS on Linux ARC is now behaving like I expect a ZFS ARC to behave in general.

(Watching 'arcstat.py 1' also suggests that the ARC is being pretty effective when I do things like compile Firefox from source.)

I don't know if all of ZFS on Linux's kernel memory issues are gone. I haven't run into any, but that isn't exactly conclusive (and I do look on my machine for signs of things like memory fragmentation). But I can say that ZoL is now handling the ARC much better than it used to and seems to be much more effective at stealing memory back from ext4. In general this strikes me as a good omen for ZoL really solving its long standing memory usage issues, which will make it a significantly safer thing to use.

Sidebar: Some numbers

Although my machine's current state isn't quite comparable to what it was when I wrote my previous entry, I'll give comparative numbers anyways. On the same 32 GB machine, the ZFS ARC is currently using 16.6 GB with 12 GB of that being file data. Assuming that all of the ZFS ARC size counts as general kernel buffer/cache in 'free' output, the ext4 page cache is using under a GB. This is quite different from the previous numbers (although looking back I'm not entirely sure I was getting the size of the ext4 page cache right).

(All ZFS memory in total, ARC and otherwise, seems to be around 17.75 GB right now. This is kernel memory allocated, not memory in use, although overall utilization is around 95% to 98%.)

(Writing this entry has made me painfully aware that I was and am somewhat fuzzy on what free's output really means and that I need to research what /proc/meminfo fields themselves mean, since free is both summarizing things and being selective.)

Written on 08 August 2015.
« One thing I now really want in ZFS: faster scrubs (and resilvers)
One potential problem with cloud computing for us is the payment model »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Aug 8 02:45:16 2015
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.