Linux cgroups offer a bunch of robust features for limiting resource usage and handling resource contention between different groups of processes, which you can use to implement things like per-user memory and CPU resource limits. On a systemd based system, which is to say basically almost all Linuxes today, systemd more or less completely owns the cgroup hierarchy and using cgroups for resource limits requires that the processes involved be placed inside relevant systemd units, and for that matter that the systemd units exist.

Unfortunately, the mechanisms for doing this are a little bit under-developed. If you're dealing with something that goes through PAM and for which putting processes into user slices based on the UID running them is the right answer, you can use pam_systemd (which we do for various reasons). If you want a different hierarchy and things go through PAM, you can perhaps write a PAM session module that does this, copying code from pam_systemd, but I don't know if there's anything for that today. If you have processes that are started in ways that don't go through PAM, as far as I know you're currently out of luck. One case that's quite relevant for us is Apache CGI processes run through suexec.

It would be nice to be able to do better, since the odds that everything that starts processes will pick up the ability to talk to systemd to set up slices, sessions, and so on for them seem rather low. Some things have specific magic support for this, but I don't think the process is very documented and I believe it requires that things change how they start programs (so eg suexec would have to know how to do this). This means that what I'm wishing for is a daemon that would be given some sort of rules and use them to move processes between systemd slices and other units, possibly creating things like user sessions on the fly. Then you could write a rule that said 'if a process is in the Apache system cgroup and its UID isn't <X>, put it in a slice in a user hierarchy'.

An extra problem is that this daemon probably wouldn't be perfect, since it would have to react to processes after they'd appeared rather than intercept their creation; some processes could slip through the cracks or otherwise do weird things. This would make it sort of a hack, rather than something that I suspect anyone would want as a proper feature.

(I don't know if a kernel LSM could make this more reliable by intercepting and acting on certain things, like setuid() calls.)

PS: Possibly the correct answer is to persuade the Apache people to make suexec consult PAM, even if the standard suexec PAM stack does nothing. Then you could in theory add pam_systemd or whatever there. It appears that Debian may have had a custom patch for this at one but I believe they gave it up years and years ago.