The state of supporting many groups over NFS v3 in various Unixes

May 2, 2016

One of the long standing limits with NFSv3 is that the protocol only uses 16 groups; although you can be in lots of groups on both the client and the server, the protocol itself only allows the client to tell the server about 16 of them. This is a real problem for places (like us) who have users who want or need to be in lots of groups for access restriction reasons.

For a long time the only thing you could was shrug and work around this by adding and removing users from groups as their needs changed. Fortunately this has been slowly changing, partly because people have long seen this as an issue. Because the NFS v3 protocol is fixed, everyone's workaround is fundamentally the same: rather than taking the list of groups from the NFS request itself, the NFS server looks up what groups the user is in on the server.

(In theory you could merge the local group list with the request's group list, but I don't think anyone does that; they just entirely overwrite the request.)

As far as I know, the current state of affairs for various Unixes that we care about runs like this:

I care about how widespread the support for this is because we've finally reached a point where our fileservers all support this and so we could start putting people in more than 16 groups, something that various parties are very much looking forward to. So I wanted to know whether officially adding support for this would still leave us with plenty of options for what OS to run on future fileservers, or whether this would instead be a situation more like ACLs over NFS. Clearly the answer is good news; basically anything we'd want to use as a fileserver OS supports this, even the unlikely candidate of Oracle Solaris.

(I haven't bothered checking out the state of support for this on the other *BSDs because we're not likely to use any of them for an NFS fileserver. Nor have I looked at the state of support for this on dedicated NFS fileserver appliances, because I don't think we'll ever have the kind of budget or need that would make any of them attractive. Sorry, NetApp, you were cool once upon a time.)


Comments on this page:

For a long time the only thing you could was shrug and work around this by adding and removing users from groups as their needs changed.

We hit this at $WORK a while ago, and some other options are:

  • sg(1) command on Linux: "execute command as different group ID"; basically the group equivalent to su(1)
  • newgrp(1): "log in to a new group"

Kludgy, but useful in some of the scripts our users used where they had to read/write into different "lab" areas.

For those keeping score, the "appliance vendors" NetApp and EMC Isilon both support "server-side lookups": the group information in the RPC call is ignored, and the UID is searched on the server itself. Fairly straight forwards with Isilon (see also user-mappings); not sure about the details with NetApp.

$WORK has a bunch of Isilon for HPC, and we've used the server-side lookup with many, many Linux NFSv3 clients for a few years now without issue (after setting up user-mappings properly). Generally don't have to use the above workarounds now.

By Robert Sander at 2016-05-03 04:48:39:

I ran into this problem years ago and found this patch for the Linux NFS client code: http://www.frankvm.com/nfs-ngroups/README

It basically decides which groups to send to the server based on the filesystem permissions.

The last update is for kernel version 4.3.

Written on 02 May 2016.
« I should keep and check notes even on my own little problems
How I think you set up fair share scheduling under systemd »

Page tools: View Source, View Normal.
Search:
Login: Password:

Last modified: Mon May 2 00:46:00 2016
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.