Modern Linux kernel memory allocation rules for higher-order page requests
Back in 2012 I wrote an entry on why our Ubuntu 10.04 server had a page allocation failure, despite apparently having a bunch of memory free. The answer boiled down to the the NFS code wanting to allocate a higher-order request of 64 Kb of (physically contiguous) memory and the kernel having some rather complicated and confusing rules for when this was permitted when memory was reasonably fragmented and low(-ish).
That was four and a half years ago, back in the days of kernel 3.5.
Four years is a long time for the kernel. Today the kernel people are working on 4.11 and, unsurprisingly,
things have changed around a bit in this area of code. The function
involved is still called
__zone_watermark_ok() in mm/page_alloc.c,
but it is much simpler today. As far as I can tell from the code, the
new general approach is nicely described by the function's current
Return true if free base pages are above 'mark'. For high-order checks it will return true of the order-0 watermark is reached and there is at least one free page of a suitable size. Checking now avoids taking the zone lock to check in the allocation paths if no pages are free.
The 'order-0' watermark is the overall lowmem watermark (which I
low: from my old entry).
This bounds all requests for obvious reasons; as the code says in
a comment, if a request for a single page is not something that can
go ahead, requests for more than one page certainly can't. Requests
for order-0 pages merely have to pass this watermark; if they do,
they get a page.
Requests for higher-order pages have to pass an obvious additional check, which is that there has to be a chunk of at least the required order that's still free. If you ask for a 64 Kb contiguous chunk, your request can't be satisfied unless there's at least one chunk of size 64 Kb or bigger left, but it's satisfied if there's even a single such chunk. Unlike in the past, as far as I can tell requests for higher-order pages can now consume all of those pages, possibly leaving only fragmented order-0 4 Kb pages free in the zone. There is no longer any attempt to have a (different) low water mark for higher-order allocations.
This change happened in late 2015, in commit 97a16fc82a; as far as I can tell it comes after kernel 4.3 and before kernel 4.4-rc1. I believe it's one commit in a series by Mel Gorman that reworks various aspects of kernel memory management in this area. His commit message has an interesting discussion of the history of high-order watermarks and why they're apparently not necessary any more.
(Certainly I'm happy to have this odd kernel memory allocation failure mode eliminated.)
Sidebar: Which distributions have this change
Ubuntu 16.04 LTS uses kernel '4.4.0' (plus many Ubuntu patches); it has this change, although with some Ubuntu modifications from the stock 4.4.0 code. Ubuntu 14.04 LTS has kernel 3.13.0 so it shouldn't have this change.
CentOS 7 is using a kernel labeled '3.10.0'. Unsurprisingly, it does not have this change and so should have the old behavior, although Red Hat has been known to patch their kernels so much that I can't be completely sure that they haven't done something here.
Debian Stable has kernel 3.16.39, and thus should also be using the old code and the old behavior. Debian Testing ('stretch') has kernel 4.9.13, so it should have this change and so the next Debian stable release will include it.
My theory on why Go's
gofmt has wound up being accepted
In Three Months of Go (from a Haskeller's perspective) (via), Michael Walker makes the following observation in passing:
I do find it a little strange that gofmt has been completely accepted, whereas Python’s significant whitespace (which is there for exactly the same reason: enforcing readable code) has been much more contentious across the programming community.
As it happens, I have a theory about this: I think it's important
gofmt only has social force. By this I mean that you can
write Go code in whatever style and indentation you want, and the
Go compiler will accept it (in some styles you'll have to use more
semicolons than in others). This is not the case in Python, where
the language itself flatly insists that you use whitespace in roughly
the correct way. In Go, the only thing 'forcing' you to put your
gofmt is the social expectations of the Go community.
This is a powerful force (especially when people learning Go also
learn 'run your code through
gofmt'), but it is a soft force as
compared to the hard force of Python's language specification, and
so I think people are more accepting of it. Many of the grumpy
reactions to Python's indentation rules seem to be not because the
formatting it imposes is bad but because people reflexively object
to being forced to do it.
(This also means that Go looks more conventional as a programming language; it has explicit block delimiters, for example. I think that people often react to languages that look weird and unconventional.)
There is an important practical side effect of this that is worth
noting, which is that your pre-
gofmt code can be completely sloppy.
You can just slap some code into the file with terrible indentation
or no indentation at all, and
gofmt will fix it all up for you.
This is not the case in Python; because whitespace is part of the
grammar, your Python code must have good indentation from the
start and cannot be fixed later. This
makes it easier to write Go code (and to write it in a wide variety
of editors that don't necessarily have smart indentation support
and so on).
The combination of these two gives the soft force of
gofmt a great
deal of power over the long term. It's quite convenient to be able
to scribble sloppily formatted code down and then have
it all nice for you, but if you do this you must go along with
gofmt's style choices even if you disagree with some of them.
You can hold out and stick to your own style, but you're doing
things the hard way as well as the socially disapproved way, and
in my personal experience sooner or later it's
not worth fighting Go's city hall any more. The lazy way wins out
gofmt notches up another quiet victory.
(It probably also matters that a number of editors have convenient
gofmt integration. I wouldn't use it as a fixup tool as much as
I do if I had to drop the file from my editor, run
gofmt by hand,
and then reload the now-changed file. And if it was less of a fixup
tool, there would be less soft pressure of 'this is just the easiest
way to fix up code formatting so it looks nice'; I'd be more likely
to format my Go code 'correctly' in my editor to start with.)