Chris's Wiki :: blog/unix/ChmodInterestingFailure

Sometimes, `chmod` can fail for interesting reasons

May 3, 2017

I'll start by presenting this rather interesting and puzzling failure in illustrated form:

; mkdir /tmp/newdir
; chmod g+s /tmp/newdir
chmod: /tmp/newdir: Operation not permitted

How can I not be able to make this chmod change when I just made the directory and I own it? For extra fun, some people on this particular system won't experience this problem, and in fact many of them are the people you might report this problem to, namely the sysadmins.

At first I wondered if this particular /tmp filesystem disallowed setuid and setgid entirely, but it turned out to be not that straightforward:

; ls -ld /tmp/newdir
drwxr-xr-x  2 cks  wheel  512 May  3 00:35 /tmp/newdir

This at least explains why my chmod attempt failed. I'm not in group wheel, and for good reasons you can't make a file setgid to a group that you're not a member of. But how on earth did my newly created directory in /tmp wind up in group wheel, a group I'm not a member of? Well, perhaps someone made /tmp setgid, so all directories created in it inherited its group (presumably group wheel). Let's see:

; ld -ld /tmp
drwxrwxrwt  157 root  wheel  11776 May  3 00:41 /tmp

Although /tmp is indeed group wheel, it has perfectly ordinary permissions (mode 777 and sticky ('t'), so you can only delete or rename your own files). There's no setgid to be seen.

The answer to this mystery is that this is a FreeBSD machine, and on FreeBSD, well, let's quote the mkdir(2) manpage:

The directory's owner ID is set to the process's effective user ID. The directory's group ID is set to that of the parent directory in which it is created.

And also the section of the open(2) manpage that deals with creation of new files:

When a new file is created it is given the group of the directory which contains it.

In other words, on FreeBSD all directories have an implicit setgid bit. Everything created inside them (whether directories or files) inherits the directory's group. Normally this is not a problem and you'll probably never notice, but /tmp (and /var/tmp) are special because they allow everyone to create files and directories in them, and so there are a lot of people making things there who are not a member of the directory's group.

(The sysadmins usually are members of group wheel, though, so things will work for them. This should add extra fun if a user reports the general chmod issue as a problem, since sysadmins can't reproduce it as themselves.)

You might think that this is an obscure issue that no one will ever care about, but actually it caused a Go build failure on FreeBSD for a while. Tracking down the problem took me a while and a bunch of head scratching.

PS: arguably GID 0 should not be group wheel but instead something else that only root is a member of and wheel should be a completely separate group. To have group wheel used for group ownership as well as su access to root is at least confusing.

Comments on this page:

By p.kern at 2017-05-03 12:17:22:

hmmm ... just on the basis of your description, it sounds like the fault is with the Go build procedure for making assumptions about the group of a directory, instead of explicitly setting the group before trying to chmod it. just my 2 bits.

By cks at 2017-05-03 12:29:24:

It's traditional and still-common Unix behavior that when you make a new directory in /tmp, it's in your group (and you can thus set it setgid). The FreeBSD behavior is allowed by the Single Unix Specification for mkdir(), but it's uncommon, so I can't blame the Go authors for missing this.

By Greg A. Woods at 2017-05-03 19:20:43:

Once upon a time I think there were a significant number of people who would have said the BSD behaviour of mkdir(2) was the "normal" way -- at least for systems allowing simultaneous multiple group-ID membership.

By cks at 2017-05-03 21:39:29:

Time for me to eat humble pie. Greg A. Woods is correct; the FreeBSD behavior is the traditional BSD behavior (as seen in eg the 4.2 BSD mkdir(2) manpage). I don't know where setgid on directories restoring the BSD behavior came from, but I would suspect System V Release 4, since SVR4 is where the great AT&T/BSD reunification mostly happened.

(This sheds an additional light on the POSIX and SUS standards allowing this behavior. If it was traditional BSD behavior, of course they had to allow it; POSIX's purpose was mostly to codify existing Unix behavior.)

By Greg A. Woods at 2017-05-04 01:05:12:

Being a former SysVR4 fan I would say that the (ab)use of the SETGID bit on directories to enable BSD semantics made good enough sense at the time if you were coming from the AT&T point of view. I seem to remember something about arguments in the security requirements circles fussing over the worry about attribution and accountability, but in the end it's the UID that is solely responsible for those aspects.

The BSD semantics are a bit weird and probably went too far -- as you sort of suggested it is a bit of a stretch to give group ownership to objects even when the creator has no obvious membership in the said group. This may be something to do with the fact that in early systems you could couldn't trust the in-kernel groups list of a process to be complete (it was limited in size), and the filesystem sure as heck can't be parsing the /etc/group file every time someone creates a new file (even caching /etc/group every time it is updated would be entirely out of the question for early unix kernel programmers).

Written on 03 May 2017.

«	When a TLS client's certificate is offered to the TLS server

My views on using LVM for your system disk and root filesystem

Sometimes, chmod can fail for interesting reasons

Comments on this page:

Sometimes, `chmod` can fail for interesting reasons