NFS's problem with (concurrent) writes
If you hang around distributed filesystem developers, you may hear them say grumpy things about NFS's handling of concurrent writes and writes in general. If you're an outsider this can be a little bit opaque. I didn't fully remember the details until I was reminded about them recently so in my usual tradition I am going to write down the core problem. To start with I should say that the core problem is with NFS the protocol, not any particular implementation.
Suppose that you have two processes, A and B. A is writing to a file and
B is reading from it (perhaps they are cooperating database processes or
something). If A and B are running on the same machine, the moment that
write() the newly-written data is visible to B when it next
read() (or it's directly visible if B has the file
Now we put A and B on different machines, sharing access to the file
over NFS. Suddenly we have a problem, or actually two problems.
First, NFS is silent on how long A's kernel can hold on to the
before sending it to the NFS server. If A
the file the kernel must ship the writes off to the NFS server, but
before then it may hang on to them for some amount of time at its
convenience. Second, NFS has no protocol for the server to notify B's
kernel that there is updated data in the file. Instead B's kernel may
be holding on to what is now old cached data that it will quietly give
to B, even though the server has new data. Properly functioning NFS
clients check for this when you
open() a file (and discard old data if
necessary); I believe that they may check at other times but it's not
The CS way of putting this is that this is a distributed cache invalidation problem and NFS has only very basic support for it. Basically NFS punts and tells you to use higher-level mechanisms to make this work, mechanisms that mean A and B have to be at least a bit NFS-aware. Many modern distributed and cluster filesystems have much more robust support that guarantees processes A and B see a result much closer to what they would if they ran on the same machine (some distributed FSes probably guarantee that it's basically equivalent).
(Apparently one term of art for this is that NFS has only 'close to open' consistency, ie you only get consistent results among a pool of clients if A closes the file before B opens it.)
Comments on this page:Written on 21 October 2013.