Wandering Thoughts archives

What the Solaris 8 nfs3_max_threads parameter probably controls

We've recently been having a problem where one specific client experiences really slow IO to one specific NFS mounted filesystem, especially for things like file creation. (Other clients can use the filesystem at full speed, and this client can use other filesystems at full speed.)

We immediately thought 'per-filesystem concurrency limits' and soon turned up the Solaris 8 NFS kernel tunable nfs3_max_threads, which the fine documentation helpfully describes as:

This symbol controls the maximum number of async threads started per file system mounted using NFS version 3 software.

That's all very well and good, but it leaves one with an important question: just what are async threads necessary for in a Solaris 8 NFS client?

The Sun documentation is no help; the only mention of async threads is in the documentation for NFS kernel tunables. (Par for the course, Google has a more useful search engine for docs.sun.com than Sun does. Also, I have to say that docs.sun.com is achingly slow.)

The online O'Reilly Managing NFS and NIS book mentions the parameter, but is of no help in clarifying things. On the one hand it says that they're only used for readahead and writeback, but on the other hand it says that the number of async threads puts a limit on how much outstanding IO a client can dump on a server per filesystem (which cannot be true unless all NFS IO goes through async threads).

Interestingly, the Solaris 9 documentation says (about the NFS v3 case):

The operations that can be executed asynchronously are read for read-ahead, readdir for readdir read-ahead, write for putpage and pageio requests, and commit.

I don't know how much has changed in this code between Solaris 8 and Solaris 9, but I think it's likely to be pretty close (and it's the best answer I'll probably get, short of a chance to spelunk the Solaris 8 kernel code). And points to Sun for writing useful documentation about it only slightly too late to be directly useful for our systems.

(Unfortunately, all of this winds up suggesting that the problem going away after we cranked nfs3_max_threads up to 32 from the default of 8 may just have been coincidence. Sometimes ignorance is bliss, or at least confidence.)