How mountd and exportfs handle NFS export permissions on Linux

July 31, 2019

While the Linux kernel NFS server maintains an authentication cache, the final authority on what what filesystems are exported to who and with what permissions is rpc.mountd. Mountd gets this information from /var/lib/nfs/etab, which conveniently is a plain text file. However, mountd reads the information from etab into an internal data structure and only re-does this when etab's inode number changes. As far as I can tell there's nothing else to it, which means that you can create a new version of etab by hand if you want to.

(While lsof will tell you that rpc.mountd has etab open, mountd does this purely so that etab's current inode number can't be reused for a new file. It never re-reads the opened file.)

Normally, new versions of /var/lib/nfs/etab are created only by exportfs (which writes the new version to an 'etab.tmp' file and then renames it). Because exportfs allows you to make NFS export changes through the command line that are not present in /etc/exports and /etc/exports.d/ files, in normal operation exportfs determines the new contents of /var/lib/nfs/etab in part by merging the current contents in with your new changes. Your new changes can come from the command line, for things like 'exportfs -u <client>:<path>' and 'exportfs -i -o <options> <client>:<path>', or from /etc/exports and company for things like 'exportfs -a'. This behavior of merging in the exports from the current etab is why 'exportfs -a' doesn't remove exports that are no longer in /etc/exports.

(A plain 'exportfs -au' has the obvious behavior of writing an empty /var/lib/nfs/etab.)

For exports that exist in both etab and exports, this merging process will replace export options from the old etab with the versions from /etc/exports and company, including things like 'ro' versus 'rw'. This means that an 'exportfs -a' will at least update client access permissions for existing exports, even if it won't cut off clients who have been entirely removed from the export permissions.

Exportfs also has a '-r' option, which is described by the manpage as:

Reexport all directories, synchronizing /var/lib/nfs/etab with /etc/exports. This option removes entries in /var/lib/nfs/etab which have been deleted from /etc/exports, and removes any entries from the kernel export table which are no longer valid.

Although the code in exportfs.c is hard to follow, the first part of 'exportfs -r' is implemented by generating the new etab purely from /etc/exports and company, without merging in the current contents of /var/lib/nfs/etab. This does exactly what you want once the kernel caches are flushed, and does it without un-exporting anything that should stay exported. If something is exported in both the old etab and the updated etab, obviously rpc.mountd will always permit access; there will never be a period where rpc.mountd is using an etab without it listed.

(The second claim in the description is what I would call not entirely correct. The actual code simply flushes the caches in general. As covered in this entry, in modern kernels any flush is a total flush, which means that 'exportfs -fr' and 'exportfs -r' do the same thing in the end. In older kernels, what gets flushed without '-f' is somewhat chancy, so I think you probably want to use '-f' to be sure.)

Back in January I mentioned 'exportfs -r' and wondered why our system for ZFS NFS export permissions wasn't using it. Given what I now know about how all of this works, we definitely should be using 'exportfs -r' instead of our current approach of first un-exporting a filesystem and then re-exporting it (and we'll be changing our scripts to implement this). 'exportfs -r' does exactly what we want when changing the NFS sharing for an existing NFS export. In general, what you want to do for seamless persistent NFS export changes is to write the new information to /etc/exports or some file in /etc/exports.d and then run 'exportfs -r' to resynchronize /var/lib/nfs/etab to your new reality.

(I believe that you want to do this even when removing an export entirely, or at least that doing so is the simplest way of un-exporting something.)

To put it another way, consistently using 'exportfs -r' turns /var/lib/nfs/etab into a processed cache instead of another source of truth. That's certainly what we want in general operation (perhaps not in emergencies, but emergencies are special cases).

Written on 31 July 2019.
« Link: A program to read AMD Ryzen RAPL information on Linux
How not to set up your DNS (part 24) »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Jul 31 22:18:39 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.