== What the Solaris 8 ((nfs3_max_threads)) parameter probably controls

We've recently been having a problem where one specific client
experiences really slow IO to one specific NFS mounted filesystem,
especially for things like file creation. (Other clients can use the
filesystem at full speed, and this client can use other filesystems at
full speed.)

We immediately thought 'per-filesystem concurrency limits'
and soon turned up the Solaris 8 NFS kernel tunable
((nfs3_max_threads)), which the [[fine documentation
http://docs.sun.com/app/docs/doc/806-0916/6ja8539hc?a=view]] helpfully
describes as:

> This symbol controls the maximum number of async threads started per
> file system mounted using NFS version 3 software.

That's all very well and good, but it leaves one with an important
question: just what are async threads necessary for in a Solaris 8
NFS client?

The Sun documentation is no help; the only mention of async threads
is in the documentation for NFS kernel tunables. (Par for the
course, Google has a more useful search engine for [[docs.sun.com
http://docs.sun.com]] than Sun does. Also, I have to say that
docs.sun.com is achingly slow.)

The online O'Reilly *Managing NFS and NIS* book [[mentions the parameter
http://www.unix.org.ua/orelly/networking_2ndEd/nfs/ch18_05.htm]], but is
of no help in clarifying things. On the one hand it says that they're
only used for readahead and writeback, but on the other hand it says
that the number of async threads puts a limit on how much outstanding
IO a client can dump on a server per filesystem (which cannot be true
unless all NFS IO goes through async threads).

Interestingly, the [[Solaris 9 documentation
http://docs.sun.com/app/docs/doc/806-7009/6jftnqsk4?a=view]] says
(about the NFS v3 case):

> The operations that can be executed asynchronously are read for
> read-ahead, readdir for readdir read-ahead, write for putpage and
> pageio requests, and commit.

I don't know how much has changed in this code between Solaris 8 and
Solaris 9, but I think it's likely to be pretty close (and it's the best
answer I'll probably get, short of a chance to spelunk the Solaris 8
kernel code). And points to Sun for writing useful documentation about
it only slightly too late to be directly useful for our systems.

(Unfortunately, all of this winds up suggesting that the problem going
away after we cranked ((nfs3_max_threads)) up to 32 from the default
of 8 may just have been coincidence. Sometimes ignorance is bliss, or at
least confidence.)