A while back I wrote about imposing temporary CPU and memory limits on a user, using cgroups and systemd's features to fiddle around with them. Since then we have wound up with a number of shared general use machines where we've decided it's wiser to impose limits on everyone all of the time, so that one person can't blow up a general use server through either excessive CPU usage or excessive memory usage. We've done this on Ubuntu 16.04 and now 18.04, and with some limitations it works well to keep our systems from having too many problems.

We've found that at least on 18.04, it's impossible to implement this without running a script at user login time (or more generally when a session is established). We run our script through pam_exec in Ubuntu's /etc/pam.d/common-session:

session optional pam_exec.so type=open_session /path/to/script.sh

The 'optional' bit here is really important. If you leave it out and you ever have an error in your script, you will have just locked everyone out of the machine (yourself included). As they say, ask me how I know (fortunately I did this on a test virtual machine, not a live server).

(Because we've found that limiting cron and at jobs to be necessary in our environment, we've also put this into /etc/pam.d/cron and /etc/pam.d/atd. This requires cron and at jobs to be in user sessions, but we were already doing that for other reasons.)

The script has to do two things. First, it has to turn on fair share scheduling if it's not already on. You have to check this on every session startup, because if all existing user sessions go away (ie there's no one logged in and so on), the whole fair share scheduling setup disappears. Because we want to limit both CPU and memory usage, we set both 'CPUAccounting=true' and 'MemoryAccounting=true' for user.slice itself, all currently existing 'user-*.slice' slices, and all currently existing 'session-*.scope' scopes. It's possible that some of this is overkill.

Second, we set appropriate per-user limits (based on the various bits of information about the size of the machine) by setting appropriate 'CPUQuota=...%' and 'MemoryLimit=...' values on 'user-${PAM_UID}.slice'. We also set a TasksMax. As we currently have our script implemented, it blindly overwrites any existing settings for the user's slice any time the user starts a new session, which has both advantages and drawbacks.

(All of this setting is done with 'systemctl --runtime set-property'.)

We've chosen to not do any of this for sessions for system users, including root. If the script sees that ${PAM_UID} is outside our regular user UID range, it does nothing, so root's logins are unrestricted. We could have implemented this in the pam.d file itself, using pam_succeed_if, but I feel that scripts are a better place for conditional logic like this if possible.

In the future, some of this may be possible to do through systemd drop-ins for user.slice and individual user slices. However, it certainly won't be as flexible as you can be in a script, especially if you want to behave differently for different UIDs and you have enough users that you don't want to create and maintain individual files for each of them. It would be nice to be able to reliably set fair share scheduling once, though, and not have to keep re-setting it through the script.