The shifting goals of our custom NFS mount authorization system

April 26, 2018

We've been doing custom authorization for NFS mounts in our overall environment for a very long time. Our most 'recent' system for this is the NSS-based custom NFS mount authorization scheme that we introduced on our original Solaris fileservers and now run on our OmniOS-based fileservers; this system has now been running for on the order of a decade. In one sense how this system operates has remained the same over that time (in that it still uses the same basic mechanics); in another sense, things have changed significantly because our goals and priorities for NFS mount authorization have changed in a decade.

In our system, NFS mount permission is based on what netgroup a machine is in but we then authenticate that the machine hasn't been replaced with an impostor before we allow the NFS mount. We have two sorts of machines that do NFS mounts from our fileservers; our own (Linux) servers, which are only on a couple of our networks, and then a number of additional machines run by other people on various of our sandbox networks. Our custom authorization systems have historically verified the identity of all NFS clients, both our machines and other people's machines, and the initial decade-ago version of the current system was no different.

However, over time we ran into issues with verifying our own servers. There were a whole collection of failure modes where some or many of our servers could get verification failures, and then the entire world exploded because NFS mounts are absolutely critical to having a working machine. At one point we made a quick pragmatic decision to temporarily disable the host verification for our own servers, and then as time went on we became more and more convinced that this wasn't just an expedient hack, it was the correct approach. These days our servers live on a machine room network where no outside machines are allowed, so if you can swap your own impostor machine in you have physical access to our machine room and we have major problems.

(Well, there are other options, but they're all about equally bad for us.)

As a result of this, we've now explicitly shifted to viewing our custom NFS mount authorization system as being just for verifying not-us machines (or more exactly, machines on networks we don't trust). This matters because those machines shouldn't be as crucially dependent on our NFS filesystems as our own servers are, and so we can afford to design a system that works somewhat differently, for example by requiring some active step by the NFS client to get a machine authenticated.

(We have a central administrative filesystem that's so crucial to our machines that most of them won't finish booting until they can mount it. No non-us machine should be so dependent on our NFS infrastructure (hopefully we aren't going to find out someday that one of them is anyway).)

Especially with security-related systems, it's probably a good idea to sit down periodically and re-validate all of your assumptions about how they need to work. It's very easy for your threat model to shift (as ours did), as well as your goals and needs. There's also the question of how much security the system has to provide, and at what cost (in potential misfires, complexity, and so on). You may find that the passage of time has changed your views on this for various reasons.

Written on 26 April 2018.
« An implementation difference in NSS netgroups between Linux and Solaris
Some notes on Firefox's current media autoplay settings »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Apr 26 01:11:45 2018
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.