The NFS re-export problem

January 22, 2009

One of the things that people traditionally ask for is an NFS server that can re-export its own NFS mounts (possibly among other things). Unfortunately this is impossible in general, and now I can explain why.

(In the old days, one reason people wanted this was because user level NFS servers were basically the only way to do user level filesystems at all, and it would be useful if your machine could transparently re-export your interesting user level filesystem to other machines. I'm not sure why people ask for this today.)

The core problem is that given only an NFS filehandle, the re-exporting server must both identify what real NFS server to pass the request to and recover the full original NFS filehandle to give that server. However, NFS filehandles are size-limited opaque blobs, all of which may be significant to the original NFS server; in general, there is no space to add your own identifier to a filehandle while being sure that you won't lose any information.

(If you are lucky, the NFS server that you are re-exporting uses short NFS v3 filehandles instead of full sized ones, leaving you enough space to glue the information you need on the front. Don't count on it, though.)

You can make up your own filehandle for every original filehandle that you re-export, but then you have to keep track of the mapping and do so in such a way that it persists over crashes and reboots. This is not impossible, but you're potentially going to be dealing with a lot of filehandles and NFS's statelessness means that you have no idea when clients have stopped using a particular filehandle.

(I suppose that this is less of an issue today, since NFS v3 filehandles are only at most 64 bytes long and disks get bigger every year; you can store a lot of mappings in a few gigabytes, and there are ways to revalidate and prune your database every so often.)


Comments on this page:

From 208.44.121.252 at 2009-01-22 08:56:15:

ok, I know re-exporting an NFS mount would be a bad thing, so what I'm suggesting shouldn't be considered feasable, or even advisable. Maybe not even possible. It could, in fact, get really ugly, but assume for a minute that you had a filesystem exported to a client. The client then symlinks every file to a local directory, completely replicating the hierarchy. Could the client then export that directory of symlinks to another client?

I'm pretty sure that it could export it and the 2nd client could get the directory list, since symlinks have their own inodes and it would be listing valid information, but I don't know about accessing the contents of the file.

Questions like this make me a bad person.

--Matt Simmons\ http://www.standalone-sysadmin.com

By cks at 2009-01-22 11:54:54:

This doesn't work, because NFS servers don't follow symlinks when opening files; that's up to the clients.

Written on 22 January 2009.
« Why high availability NFS requires shared storage
The HTML tax (in Python, and in general) »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Jan 22 00:36:18 2009
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.