Why NFS filehandles fail as access capabilities
I think it's clear that Sun initially intended NFS filehandles to serve as capabilities, ie opaque tokens that could be used to grant access to useful things like your files. Unfortunately, pretty soon it was clear that this didn't work and NFS filehandles actually made pretty terrible access capabilities.
(I may be mis-using the term 'access capabilities' here. As I learned it, it refers to opaque (to the user) tokens or identifiers that you can present to services to get access to some resource without having to go through some sort of more elaborate access authentication. Web session identifiers are one example of access capabilities.)
NFS filehandles failed as the NFS security mechanism because they lack at least two of the minimum properties you want in access capabilities. First, NFS filehandles were in practice guessable. This is very bad, because capabilities are only even theoretically secure if they cannot be created by the user and thus possession of a capability means that it has been given to you by the server.
Second, NFS filehandles are in practice not revocable, especially not easily, because a significant amount of the filehandle is a stable identifier for the file; to revoke a filehandle, you mostly have to create a different file and remove the old one, which can be very difficult if the file is, for example, the root directory of a filesystem. Lack of revocation means that once an NFS filehandle is compromised or guessed you can't fix the situation short of quite drastic action.
(Revocation is important in real security systems because sooner or later, something always goes wrong and an access capability leaks into the wrong hands. For good security, you need to be able to fix that without blowing up the world.)