Wandering Thoughts archives

2017-05-05

Digging into BSD's choice of Unix group for new directories and files

I have to eat some humble pie here. In comments on my entry on an interesting chmod failure, Greg A. Woods pointed out that FreeBSD's behavior of creating everything inside a directory with the group of the directory is actually traditional BSD behavior (it dates all the way back to the 1980s), not some odd new invention by FreeBSD. As traditional behavior it makes sense that it's explicitly allowed by the standards, but I've also come to think that it makes sense in context and in general. To see this, we need some background about the problem facing BSD.

In the beginning, two things were true in Unix: there was no mkdir() system call, and processes could only be in one group at a time. With processes being in only one group, the choice of the group for a newly created filesystem object was easy; it was your current group. This was felt to be sufficiently obvious behavior that the V7 creat(2) manpage doesn't even mention it.

(The actual behavior is implemented in the kernel in maknode() in iget.c.)

Now things get interesting. 4.1c BSD seems to be where mkdir(2) is introduced and where creat() stops being a system call and becomes an option to open(2). It's also where processes can be in multiple groups for the first time. The 4.1c BSD open(2) manpage is silent about the group of newly created files, while the mkdir(2) manpage specifically claims that new directories will have your effective group (ie, the V7 behavior). This is actually wrong. In both mkdir() in sys_directory.c and maknode() in ufs_syscalls.c, the group of the newly created object is set to the group of the parent directory. Then finally in the 4.2 BSD mkdir(2) manpage the group of the new directory is correctly documented (the 4.2 BSD open(2) manpage continues to say nothing about this). So BSD's traditional behavior was introduced at the same time as processes being in multiple groups, and we can guess that it was introduced as part of that change.

When your process can only be in a single group, as in V7, it makes perfect sense to create new filesystem objects with that as their group. It's basically the same case as making new filesystem objects be owned by you; just as they get your UID, they also get your GID. When your process can be in multiple groups, things get less clear. A filesystem object can only be in one group, so which of your several groups should a new filesystem object be owned by, and how can you most conveniently change that choice?

One option is to have some notion of a 'primary group' and then provide ways to shuffle around which of your groups is the primary group. One problem with this is that it's awkward and error-prone to work in different areas of the filesystem where you want your new files and directories to be in different groups; every time you cd around, you may have to remember to change your primary group. If you move into a collaborative directory, better shift (in your shell) to that group; cd back to $HOME, or simply want to write a new file in $HOME, and you'd better remember to change back.

Another option is the BSD choice of inheriting the group from context. By far the most common case is that you want your new files and directories to be created in the 'context', ie the group, of the surrounding directory. If you're working in $HOME, this is your primary login group; if you're working in a collaborative area, this is the group being used for collaboration. Arguably it's a feature that you don't even have to be in that group (if directory permissions allow you to make new files). Since you can chgrp directories that you own, this option also gives you a relatively easy and persistent way to change which group is chosen for any particular area.

If you fully embrace the idea of Unix processes being in multiple groups, not just having one primary group and then some number of secondary groups, then the BSD choice makes a lot of sense. And for all of its faults, BSD tended to relatively fully embrace its changes (not totally, perhaps partly because it had backwards compatibility issues to consider). While it leads to some odd issues, such as the one I ran into, pretty much any choice here is going to have some oddities. It's also probably the more usable choice in general if you expect much collaboration between different people (well, different Unix logins), partly because it mostly doesn't require people to remember to do things.

(I know that on our systems, a lot of directories intended for collaborative work tend to end up being setgid specifically to get this behavior.)

BSDDirectoryGroupChoice written at 01:00:53; Add Comment

2017-05-03

Sometimes, chmod can fail for interesting reasons

I'll start by presenting this rather interesting and puzzling failure in illustrated form:

; mkdir /tmp/newdir
; chmod g+s /tmp/newdir
chmod: /tmp/newdir: Operation not permitted

How can I not be able to make this chmod change when I just made the directory and I own it? For extra fun, some people on this particular system won't experience this problem, and in fact many of them are the people you might report this problem to, namely the sysadmins.

At first I wondered if this particular /tmp filesystem disallowed setuid and setgid entirely, but it turned out to be not that straightforward:

; ls -ld /tmp/newdir
drwxr-xr-x  2 cks  wheel  512 May  3 00:35 /tmp/newdir

This at least explains why my chmod attempt failed. I'm not in group wheel, and for good reasons you can't make a file setgid to a group that you're not a member of. But how on earth did my newly created directory in /tmp wind up in group wheel, a group I'm not a member of? Well, perhaps someone made /tmp setgid, so all directories created in it inherited its group (presumably group wheel). Let's see:

; ld -ld /tmp
drwxrwxrwt  157 root  wheel  11776 May  3 00:41 /tmp

Although /tmp is indeed group wheel, it has perfectly ordinary permissions (mode 777 and sticky ('t'), so you can only delete or rename your own files). There's no setgid to be seen.

The answer to this mystery is that this is a FreeBSD machine, and on FreeBSD, well, let's quote the mkdir(2) manpage:

The directory's owner ID is set to the process's effective user ID. The directory's group ID is set to that of the parent directory in which it is created.

And also the section of the open(2) manpage that deals with creation of new files:

When a new file is created it is given the group of the directory which contains it.

In other words, on FreeBSD all directories have an implicit setgid bit. Everything created inside them (whether directories or files) inherits the directory's group. Normally this is not a problem and you'll probably never notice, but /tmp (and /var/tmp) are special because they allow everyone to create files and directories in them, and so there are a lot of people making things there who are not a member of the directory's group.

(The sysadmins usually are members of group wheel, though, so things will work for them. This should add extra fun if a user reports the general chmod issue as a problem, since sysadmins can't reproduce it as themselves.)

You might think that this is an obscure issue that no one will ever care about, but actually it caused a Go build failure on FreeBSD for a while. Tracking down the problem took me a while and a bunch of head scratching.

PS: arguably GID 0 should not be group wheel but instead something else that only root is a member of and wheel should be a completely separate group. To have group wheel used for group ownership as well as su access to root is at least confusing.

ChmodInterestingFailure written at 01:39:47; Add Comment

By day for May 2017: 3 5; before May; after May.

Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.