Illustrating the Ubuntu clown car, AccountsService edition

August 23, 2012

AccountsService is a freedesktop.org thing to let other programs get (and set, with appropriate magic PolicyKit permissions) various information about user accounts over DBus (because everything has to go over DBus these days). The package's major component is accounts-daemon, which implements said DBus service. Ubuntu 12.04 ships with a modified version of accountsservice 0.6.15.

(By the way, it also theoretically lets other programs create, delete, and modify system properties of logins. If this is not alarming you, you are probably not a sysadmin.)

As shipped by freedesktop.org, accounts-daemon already has one problem; whenever the daemon starts up, it scans almost all accounts in /etc/passwd and gets the (full) group membership list for each of them, rereading and reparsing /etc/group for each account. Large sites have serious problems with this (also) (this bug report was found by my co-worker). Fortunately for us we only have a 500 line /etc/group and an 1800 line /etc/passwd, and it turns out that on a modern machine parsing a 500-line file over 1700 times is barely noticeable.

(This problem is particularly striking because it isn't necessary. Accounts-daemon is doing all of this work just to see if an account is in group wheel, and for that there is a much faster check that does not need to find the account's full group list.)

But this wasn't good enough for Ubuntu. Two of the account properties that you can get or set are the account's language and its locale (okay, the Ubuntu version adds the locale). In the stock code, these have no default value; if they have not been set explicitly over DBus by an outside program, they're null. In Ubuntu 12.04, Ubuntu decided that this wasn't good enough. If either is unset and you ask for its value the Ubuntu code tries to guess their correct value by opening and crudely scanning ~<user>/.profile (assuming that ~<user>/.pam_environment doesn't exist) and then falls back to /etc/default/locale. Once the code finds a value (from either source), it carefully validates the value by running an external program; in the case of the user's language, this external program is a shell script that runs a Perl program (among other things).

(See /usr/share/language-tools/language-validate. I am not making this up.)

By the way, none of this is cached. For example, the code is perfectly happy to read /etc/default/locale (and then validate the result) several thousand times.

Let's set aside everything that's wrong about crudely, unconditionally, and blindly scanning a .profile (especially since the code to do it is very limited and seems broken), because it gets worse. Ubuntu has set up its Unity environment such that the program responsible for the Unity tools and logout menu asks accounts-daemon for the locale and language of every known user when you log in (and possibly at other times too). Let me assure you that opening and reading (almost) every user's .profile (twice) takes a noticeable amount of time even with only 1700 or so real users; in fact in our fileserver environment it generally takes at least several minutes. Under some circumstances it also puts a visible load on the system (and eventually the NFS servers).

(Oh, and until this scan finishes the tools and logout menu is empty. This is somewhere between annoying and disconcerting to users, and leaves them unable to log out cleanly.)

While the same modifications to the accountsservice source package are in current versions of Debian testing, it's clear that they come from Ubuntu and were propagated into Debian. This really is an all-Ubuntu show; these modifications aren't in the upstream code and Ubuntu did not get them from someone else, they developed all of this mess themselves.

Sidebar: how bad the scan of .profile is

If you have the Ubuntu 12.04 source package for accountsservice unpacked somewhere, look at src/user.c's user_get_profile_env function. Simplified and with a bit of pseudo-code, this does:

char line[50];
fp = fopen(profile_path, "r");
while ((fgets(line, 50, fp)) != NULL) {
  if (line starts with 'export LANGUAGE="') {
    ... extract the value
  }
  if (line starts with 'export LANG="') {
    ... extract the value
  }
}

I wish I was making this up. In low level mechanical flaws, I'm pretty convinced that this potentially breaks in interesting ways if you have lines in your .profile that are longer than 49 characters and it ignores any 'export ...' lines that are indented.


Comments on this page:

From 71.70.243.142 at 2012-08-24 00:29:57:

And what if your "export LANG=..." is in a conditional expression? My .profile needs to work on multiple Unixen/distributions with inconsistent LANG= support, so it tests its environment. Thanks for raising the alarm.

By cks at 2012-08-24 02:02:28:

Conditional expressions and similar things in .profile is one issue. Another is that there are plenty of shells (including Bash) that don't necessarily use .profile at all. The Ubuntu change assumes that everyone uses a Bourne shell and puts their locale and language initialization in .profile (if anywhere). If you use a different shell but have an old .profile (one that's ignored by your current shell), I imagine that comedy can really result.

From 207.96.159.2 at 2012-08-29 11:55:05:

With such a large number of users, I'm curious how you manage them and synchronize it with your other servers. I'm in the process of setting up a new data center and one of the issues is whether to stick with traditional password and group files or move to a directory such as LDAP.

By cks at 2012-08-29 13:42:32:

The short answer is that we've stayed with traditional passwd and group files and built a custom infrastructure to propagate them around from a master server and do various sorts of mangling. If you're interested in the details they're mostly in OurPasswordPropagation.

My personal view is that LDAP is over-complicated if all you need is replicated passwd/group files, which is pretty much our case. Also, for a data center (or other server environment) my view is that machines should be able to do as much authentication as possible even if central servers are down, because sooner or later you will really want to log into some machine when a switch has died or your central (LDAP) server has exploded or whatever.

From 96.42.195.111 at 2013-04-06 19:25:52:

How terrible. You should ask for your money back.

By cks at 2013-04-06 19:40:19:

Open source does not excuse incompetence, especially when groups present themselves as competent and trustworthy and then hack (badly) on core components of something important.

By philnc at 2017-06-20 09:05:56:

Found this article while scouring the Internet for information on AccountsService, yet another poorly documented software on freedesktop.org, in connection with an issue I was having on Fedora 25. The insanity that is accountsservice is not just a Ubuntu thing: it also infects the Red Hat ecosystem, including the latest releases of Fedora and RHEL.

Thanks for this post, it helped me sort out some of what I was seeing.

Written on 23 August 2012.
« My view on the understandability of language idioms
The theoretical right way to check if an account is in a Unix group »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Aug 23 01:56:04 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.