The Linux kernel, simultaneous multithreading, and process scheduling

January 27, 2022

Back in an earlier entry on simultaneous multithreading, I said that I expected operating systems, Linux included, to generally schedule processes on to a single CPU of each core before it started doubling up processes on two CPUs of a single core. This raises the question of whether the Linux process scheduler is SMT-aware and what it seems to do in practice.

The answer to the first question is that it clearly is SMT aware, although I haven't traced through the code to see exactly what effects it has. There is a SCHED_SMT kernel configuration option that affects various things, and there are various comments about this (and code) in kernel/sched/fair.c and kernel/sched/topology.c, among other spots. The comments I've skimmed through suggest that the kernel does all of the obvious things with SMT pairs, like considering the caches 'hot' for a process on both CPUs if it was running recently on one of them.

For how things go in practice, a fairly current Linux kernel (the Fedora 34 version of 5.15.15) running one CPU consuming process per core (not CPU) seems to mostly distribute CPU load the way I'd expect. Using tools like htop and mpstat doesn't let me see the CPU (or core) scheduling history of individual processes, but the kernel doesn't do anything obviously weird looking. CPU utilization does hop from CPU to CPU from time to time, which I suspect is an artifact of other running processes preempting the CPU hogs off their original CPU and on to the other CPU for that core. A CPU load that makes a bunch of system calls (basically an infinite loop in the shell) looks more erratic in htop; the CPU load appears to bounce around a lot and there's a bunch of system time involved too.

My conclusion from looking is that the Linux kernel in a normally operating system doesn't do anything glaringly obvious about what CPU gets used across cores. For instance, it's not like the first CPU of each core gets used almost all of the time and the load only spills over to the second CPU during high load periods. Even with very low load, every CPU can get used from time to time (you can see this in both htop and mpstat).

(In short, it's all boring, with no surprises or interesting things that I could see.)

Written on 27 January 2022.
« Django and Apache HTTP Basic Authentication (and REMOTE_USER)
Some things on Django's CSRF protection, sessions, and REMOTE_USER »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Jan 27 23:23:57 2022
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.