The problem of Ubuntu 20.04, Snaps, and where your home directory is

April 29, 2020

I tweeted:

Due to Chromium becoming a Snap in Ubuntu 20.04 and our user home directories not being under /home, it looks like our users will have to live without Chromium. Thanks, Ubuntu.

This may sound a little bit hard to believe, so let me show you the stages of trying to use chromium-browser on 20.04 (installed by doing 'apt-get install chromium-browser'):

; chromium-browser
2020/04/30 00:44:29.065359 cmd_run.go:560: WARNING: XAUTHORITY environment value is not a clean path: "/h/281/cks/.Xauthority"
 cannot open path of the current working directory: Permission denied
; cp -a $HOME/.Xauthority /tmp/cks-xauth; export XAUTHORITY=/tmp/cks-xauth
; chromium-browser 
cannot open path of the current working directory: Permission denied
; cd /tmp; chromium-browser
Sorry, home directories outside of /home are not currently supported. 
See for details.

On Ubuntu 20.04, Canonical delivers 'chromium-browser' (the open source version of Chrome) as a Snap, which is the jargon name for a 'Snappy', well, thing. The installed and visible /usr/bin/chromium-browser is just a shell script that flails around and eventually runs /snap/bin/chromium, which is a symlink to /usr/bin/snap, which magically winds up running Chromium as a sandboxed and oddly set up program. Or trying, at any rate; it fails, as you can see, because our long standing system configuration is incompatible with how Canonical requires you to operate your Ubuntu systems now.

(It would be nice if the whole system told you the core problem right away and saved you the initial attempts to make things work.)

Although the messages do not say this, the Snap system ignores any $HOME environment variable that you might have set; what it cares about is where /etc/passwd says your home directory is (after any symlinks are resolved). The official documentation says 'a workaround is to bind mount the home directory outside /home into /home'. We currently have roughly 2600 user home directories that are not under /home; instead they are under an assortment of filesystems that are NFS mounted from our Linux ZFS fileservers. Bind mounting all of their filesystems into /home or changing all of them to be NFS mounted there is what we can politely call not feasible, and certainly massively disruptive even if it was feasible.

Given this, I have to say that Canonical's Snappy system does not appear to be designed for anything except small environments, such as desktops and laptops used by one or a few people. It's certainly not viable in our larger (but still small scale) multi-user Unix environment. At the moment, we can probably live without Chromium in 20.04, although some of our users are going to be upset. If Canonical packages more important things as Snaps instead of actual packages, we will probably be forced to do something much more drastic.

(One quite likely option is migrating from Ubuntu LTS to Debian, who will probably never be this narrowly focused on small desktops or so willing to throw away compatibility with different environments.)

Comments on this page:

I have to say that Canonical's Snappy system does not appear to be designed for anything except small environments, such as desktops and laptops used by one or a few people.

I'm "one or a few people" and I don't like the idea of snaps. To me it seems like another shiny new thing trying to "fix" something that isn't broken. What's wrong with "apt-get install"?

From at 2020-04-30 09:33:02:

Just out of curiosity, is symlinking your "$HOME" into "/home" enough to trick chromium into working? If not, why?

By Alex Xu at 2020-04-30 10:18:10:

this seems like it could be worked around reasonably easily with pam_mount. you wouldn't need to pre-compile a list of all users that might log in, and installing and testing a new software isn't so bad if you're already upgrading the whole OS anyways.

By cks at 2020-04-30 12:00:43:

I don't believe that symlinks will work, because the underlying problem is that snaps are sandboxed, with limited access to areas of the filesystem. So I expect that your home directory must be under /home as the kernel sees it, which is after symlinks are resolved. Bind mounts will do this; other mechanisms mostly not.

As far as pam_mount goes, it's not just the /home mount that matters, it's also what /etc/passwd says. Changing user real home directories to appear in /home is a disruptive change that affects everyone who previously used other paths, as well as programs that 'known' where your home directory is (or was), and it would require significant changes to our overall passwd distribution system. Also, our login servers routinely have upwards of 40 different people logged in. We're not willing to do this much work and go through this much disruption for Chromium, or indeed for any Snaps.

By Alex Xu at 2020-04-30 23:44:04:

I interpret "A workaround is to bind mount the home directory outside /home into /home." to mean that if your home directory is in /usr/home/myuser, then snap will work if you execute "mount --bind /usr/home/myuser /home/myuser" before running the program.

Assuming this is the case, pam_mount is perfectly capable of performing this task upon log in based on the user's actual registered home directory.

Linux is also perfectly capable of handling large numbers of mounts. The default maximum is apparently 100000. With 100 mounts, each mount/umount takes about 1ms, which ought to be negligible if you're using Ubuntu anyways. With 10000 mounts, the userspace utilities stop working properly though. If you have 10000 simultaneous active logins on a single machine, probably nothing else will work anyways, so it's a moot point.

I agree that snaps are terrible, but this particular problem seems reasonably easy to work around. It's possible (probable) that there are other issues, the sum of which amount to "not worth it anymore", but that doesn't bear on whether this particular issue is a dealbreaker.

By Andrea Borghi at 2020-05-01 08:30:46:

Extending Peter Donis comment, this is a real shame that distributions and upstream software maintainer are forcing systems administrators to not use software or radically change their systems on the basis on a perceived idea of how the systems should run, idea that can not be universal as Chris case has demonstrated.

another two examples are VLC and systemd on debian.

the first refuses to run as root, but we are on a mini sistems of mine where there are no users and it is used to convert video content in a file share to a multicast flow, for a distributed display in a commercial setting (resolved by recompliling VLC without the control)

Or Systemd, which has the same problem, by forcing services to run anywere but /home. I historically have the /home mounted from fileservers and start the services under the home of the respective users/service. And then? the new shiny thing is use the ProtectHome on the standard unit delivered in install packages, so the started service starts in a namespace without a readable /home and without its files (this is maxed when instead of copying the systemd unit to /etc/systemd/system; i used the override mechanism of systemd... where i personalized only some setting and all was well... until upstream inserted the protecthome in an update and magically my service does not start anymore, so i returned to copying the entire unit and disabling the oj so automagically magical mechanism of systemd.)

It seems to me that the current state is what was predicted at systemd inception, as time passes more and more highly intelligent developers are forcing all the world (and all the complexity it represents) in a shoebox.

And treating all use cases as they were theis precious notebook.

This is not not not good.

not good as nowadays my worktime is spent more into removing these absurd and artificial limitations than doing my job.

Let's see the snap packages or docker containers from a high level perspective. They are good at isolating and sandboxing, and they are good at gruping all the dependencies of an application with that application.

but but we are replicating also the basic libraries so we are witnessing the birth of another dll hell, because the upstream developers will not track of all the updates and security fixes on dependencies of their applications so we end up with multiple versions of libraries, tools, applications of dubious updating schedule and affidability. In small words, a very big mess

By cks at 2020-05-01 12:05:37:

Alex Xu: if bind mounts into /home work in general, they definitely don't work for NFS mounted home directories; attempts eventually fail with mysterious errors once you hack up enough of your system to pacify Snappy. Also, you definitely do have to change what /etc/passwd lists as people's home directories; otherwise, things simply don't get off the ground (even though other bits trace through symlinks and print real path names).

Since bind mounts fail, I expect that mounting NFS filesystems under /home won't work either; the two are almost the same from the kernel's point of view.

(And since Snappy evidently has hard-coded checks for /home that require an /etc/passwd entry that points there, merely changing the Apparmor profile is also clearly not enough. We are talking major system mutilation to even theoretically get this working, with no assurance that it really will.)

By Matt at 2020-05-01 12:21:23:

Wait, on your systems what $HOME says is my home directory and what /etc/passwd says is my home directory are two different things???

By cks at 2020-05-01 12:40:50:

Not normally, but many things will use $HOME over /etc/passwd so you can do testing with things like 'mkdir /tmp/test; HOME=/tmp/test chromium-browser'. This doesn't work with Snaps; they appear to care about /etc/passwd only and ignore $HOME.

Written on 29 April 2020.
« Dealing with my worries about Django and HTTP Basic Authentication
The afterlife of Python 2 »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Apr 29 21:18:02 2020
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.