Allowing people to be in more than 16 groups with an OmniOS NFS server

April 14, 2015

One of the long standing problems with traditional NFS is that the protocol only uses 16 groups; although you can be in lots of groups on the client (and on the server), the protocol itself only allows the client to tell the server about 16 of them. Recent versions of Illumos added a workaround (based on the Solaris one) where the server will ignore the list of groups the client sent it and look up the UID's full local group membership. Well, sometimes it will do this, if you get all of the conditions right.

There are two conditions. First, the request from the client must have a full 16 groups in it. This is normally what should happen if GIDs are synchronized between the server and the clients, but in exceptional cases you should watch out for this; if the client sends only 15 groups the server won't do any lookups locally and so can deny permissions for a file you actually have access to based on your server GID list.

Second and less obviously, the server itself must be explicitly configured to allow more than 16 groups. This is the kernel tunable ngroups_max, set in /etc/system:

set ngroups_max = 64

Any number larger than 16 will do, although you want it to cover the maximum number of groups you expect people to be in. I don't know if you can set it dynamically with mdb, so you probably really want to plan ahead on this one. On the positive side, this is the only server side change you need to make; no NFS service parameters need to be altered.

(This ngroups_max need is a little bit surprising if you're mostly familiar with other Unixes, which generally have much larger out of the box settings for this.)

This Illumos change made it into the just-released OmniOS r151014 but is not in any earlier version as far as I know. Anyways, r151014 is a LTS release so you probably want to be using it. I don't know enough about other Illumos distributions like SmartOS and Nexenta's offering to know when (or if) this change made it into them.

(The actual change is Illumos issue 5296 and was committed to the Illumos master in November 2014. The issue has a brief discussion of the implementation et al.)

Note that as far as I know the server and the client do not need to agree on the group list, provided that the client sends 16 groups. My test setup for this actually had me in exactly 16 groups on the client and some additional groups on the server, and it worked. This is a potential gotcha if you do not have perfect GID synchronization between server and client. You should, of course, but every so often things happen and things go wrong.

Written on 14 April 2015.
« One speed limit on your ability to upgrade your systems
Illusory security is terrible and is worse than no security »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Apr 14 00:35:51 2015
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.