Chris's Wiki :: blog/tech/NFSVsNFSWithKerberos

NFS with Kerberos and NFS without Kerberos are two quite different things

May 21, 2023

One of the big divisions between NFS v3 and NFS v4 is that NFS v4 is normally used with Kerberos. In normal configurations, this means that Kerberos is used to authenticate user requests; a NFS v4 client that claims to be making a NFS request on behalf of a given user must prove this by presenting an appropriate Kerberos ticket. This is different from NFS v3, where normally NFS clients are trusted by the NFS server to identify which UID the client is making the NFS request on behalf of. A while back I expressed an opinion about this on the Fediverse:

Hot take on NFS: NFS with Kerberos and NFS without Kerberos are two different things that aren't all that comparable, and Kerberos for NFS is a very limited fix for a very limited vulnerability that doesn't apply to most people.

(NFS with Kerberos could in theory be used for wide-area file sharing access, but in practice I believe this is almost never used. Especially over the Internet.)

In my view, the primary threat this form of Kerberos protects you from is untrusted single-user machines that claim the user has logged in to them when the user hasn't. If you trust the machines, you can trust their claims of what user they're acting on behalf of. If you don't trust a machine and people log into it, it's game over for every user that logs into the machine; the machine acquires access to their Kerberos information and can now make whatever NFS requests in their name that it wants to. If you can't trust a multi-user machine, you shouldn't allow people to log into it. So the primary threat you're protecting against is an untrusted machine operated by user A claiming that it's actually acting on behalf of user B, when user B has never touched the machine.

This is a common threat profile in the individual laptop and desktop space. But very few people are trying to use NFS in that environment (and for good reason); most commonly people use SMB/CIFS there. NFS today is mostly used between servers, where this threat is mostly not applicable.

Why this form of NFS with Kerberos isn't particularly comparable to NFS without it is that this form of NFS with Kerberos requires you to give up essentially all unattended operations that NFS clients might perform on behalf of users. Crontab entries, web CGIs or long running web server processes (including ones you want to use to get around file access permissions), server side filtering in your mail system, sufficiently long running compute jobs in your SLURM compute cluster, automatically distributed containerized workloads, generally none of those work because none of them are in a position to get the necessary Kerberos authentication from the actual person involved. All of them have to say 'trust me that I'm acting on behalf of this person', and in the usual form of NFS with Kerberos the NFS server's answer is 'no I don't trust you, prove it'.

NFS without Kerberos can be used on NFS clients as more or less just another Unix filesystem, although sharing a single area between NFS clients may require some additional work. The normal form of NFS with Kerberos can't be used that way because to a large extent it can't be used for what we could call 'background' activities, only for 'foreground' ones that happen when people are actively logged in. In many environments, ours included, these 'background' activities are quite important for the overall system, which means a switch from NFS without Kerberos to the common form of NFS with Kerberos would be a serious loss of capabilities.

(If you do face the threat that NFS with Kerberos is good at defeating, then you probably have no choice; you can't use NFS without Kerberos.)

Comments on this page:

From 193.219.181.219 at 2023-05-22 01:27:06:

But very few people are trying to use NFS in that environment (and for good reason); most commonly people use SMB/CIFS there.

I think this needs more elaboration.

I wouldn't say NFS is great over WAN, but it doesn't seem any worse than SMB if both are set up to use the same Kerberos authentication, especially with NFSv4 running entirely over the single tcp2049 (no sideband protocols). Locking works differently in NFSv4, with "leases" (which I believe are inspired by SMBv2/3).

that this form of NFS with Kerberos requires you to give up essentially all unattended operations that NFS clients might perform on behalf of users. Crontab entries, web CGIs or long running web server processes

There's a middle ground to this – it requires you to give up operations on arbitrary users, but it's always possible to have specific users to store their credentials on the system. For userspace tools with MIT Kerberos libraries there are at least five ways to set up a cronjob that authenticates on your behalf using your stored credentials (keytab), half of them are also applicable to kerberized NFS. (It's really not a new thing, other sites and universities have been doing it.)

So although you can't have a webapp that fully bypasses access permissions, you can certainly have cronjobs that run indefinitely if they're set up by their owners.

(I've read about some sites having a system that automatically creates "batch" credentials instead of the user having to store their own; IIRC Rutgers had such a system.)

By Anonymous at 2023-05-22 07:38:09:

A real benefit of NFSv4 with Kerberos that you seem to ignore, is that you can encrypt the traffic; without Kerberos (and with NFSv3) all traffic - including but not limited to passwords - is send as readable plain text.

By cks at 2023-05-22 13:13:13:

In a modern context, there are a lot of ways of securing and encrypting NFS traffic itself; as a side effect, many of these ways will also authenticate the machines involved. If you need to secure NFS against an untrusted network, you have plenty of better options than Kerberos ('better' in the sense that they have fewer drastic side effects and dependencies).

By Anonymous at 2023-05-22 13:46:27:

In a modern context, there are a lot of ways of securing and encrypting NFS traffic itself

< - snip - >

If you need to secure NFS against an untrusted network, you have plenty of better options than Kerberos

Apparently, i have not been keeping up with modern technology. I would love to read a post here about those better alternatives.

By cks at 2023-05-22 18:22:58:

My short version is that IPSec or any VPN technology is probably what you want. In a Linux environment today, I would use Wireguard. IPSec will let you encrypt only NFS traffic with native features; for other things, you'd encrypt only NFS traffic by giving the NFS servers (and clients) private 'inside' IPs and using them for NFS mounts and permissions. An extremely discount version could probably be built using SSH connection tunneling.

(Wireguard reportedly can have very good performance and high bandwidth, but I haven't tested this. Since encryption is encryption, you're probably not worse off performance wise than you would be with fully encrypted Kerberos NFS.)

My view on people who need extremely low latency and extremely high bandwidth, beyond what these solutions can provide, is that they need to build physically secure networks for their NFS traffic.

From 193.219.181.219 at 2023-05-23 00:52:58:

My short version is that IPSec or any VPN technology is probably what you want. In a Linux environment today, I would use Wireguard. IPSec will let you encrypt only NFS traffic with native features; for other things, you'd encrypt only NFS traffic by giving the NFS servers (and clients) private 'inside' IPs and using them for NFS mounts and permissions. An extremely discount version could probably be built using SSH connection tunneling.

This makes me wonder, what is it about file-access protocols that they fundamentally "have to" go through a VPN or be tunneled through SSH, in your opinion, instead of using protocol-integrated security like everything else can? (This is in context of workstation-to-server, not server-to-server.)

That is, why we use SSH-over-Internet instead of Telnet-over-VPN, or for example IMAPS-over-Internet instead of IMAP-over-VPN, trusting their built-in encryption and authentication, but reject the same thing in NFS or SMB?

(This is broadly related to "[...] in the individual laptop and desktop space. But very few people are trying to use NFS in that environment (and for good reason); most commonly people use SMB/CIFS there"; you never mentioned what specific reason it was, though.)

By cks at 2023-05-23 22:28:31:

This is a good question (set of questions) and my answers are sufficiently long that I put them in an entry, ProtocolsAndEncryption. The short version is that NFS never adopted TLS (although that's changing now) and I'm not enthused about bespoke cryptographic protocols because their track record is generally pretty bad.

One reason that NFS wasn't and isn't used anywhere near as much in the desktop/laptop space is that Unix fell out of favour there. The dominant desktop and laptop systems talk SMB/CIFS and not NFS, and SMB was rather earlier to distrusting the client machine than NFS was. NFS also spent a long time with an assortment of reliability issues in anything that wasn't an excellent network environment (issues it still has to some degree today in NFS v3; you can still get stuck locks).

Written on 21 May 2023.

«	Some notes on the cost of Go finalizers (in Go 1.20)

What I see as good options today for encrypted NFS