Wandering Thoughts archives

connect() plus write() versus sendto() for UDP sockets

The README that I wrote for my Go version of netcat contains a little aside about the 'listen on a UDP socket' mode of each program:

This is how 'nc -lu' behaves for one connection, but the mechanics are different. nc connect()s its UDP socket and then write()s to it; call leaves its UDP socket unconnected and uses sendto(). [...]

(It turns out that what netcat does is version dependent.)

So, does this make any difference? It turns out that the answer is yes, but not in a way that I expected and not really for writing.

First let's talk about write() versus send(), because sendto() is just send() with a destination address. The functional difference between the two is that send() will allow you to set various message flags. In practice there is exactly one such flag that is both useful on UDP sockets and portable: MSG_DONTROUTE, which means that your UDP packet should not be routed.

(If you're writing Linux specific code there are a number of additional flags that may be useful.)

What netcat does when it listens to a UDP socket is actually a little bit weird, so I want to describe it. Having set up the socket with bind() it does a recvfrom(..., MSG_PEEK), takes the address of the other end from the result, connect()s the UDP socket to it, and then winds up immediately calling recvfrom() again (as part of a general activity loop). When it reads things from standard input it will write() them to the (now-connected) UDP socket.

Connecting a bound UDP socket this way has two effects. The first is that you can use write() or a plain send() on the socket. The second is that a connected UDP socket stops accepting packets from other UDP sources, so netcat has converted its listening server socket into a single-connection socket. This is fine for netcat's use but would be bad if you wanted to reuse the socket as a server socket again, since there is no way to un-connect a UDP socket again.

(I suspect that this is done partly so that netcat can use the same core routine to copy things back and forth regardless of whether it's a client or a server and regardless of whether it's using TCP or UDP.)

An un-connected UDP socket used with full recvfrom() and sendto() multiplexes all client communication through the same socket. This is what you need to handle multiple clients (especially concurrently), but as far as I can see it has the drawback that you don't get any errors if the other end quietly goes away (even if the kernel could tell you, for example if the other end is local). Connected UDP sockets will report an error (eventually) under similar circumstances. And of course you need a completely different set of code than for TCP sockets or connected UDP sockets and you'll need to explicitly consider how you'll deal with multiple clients that are all active at once (since, unlike with TCP, you can't ignore all but one socket and let the kernel sort it out for you).

(This is the kind of thing I write up because it makes me do the research, so I can get this down in my own head. For someone who has a more than casual involvement with networks and networking code, I'm sometimes shockingly ignorant on the fine details of the Unix socket API. Probably I should get and read a copy of the classic Stevens work, although there may be a more modern book on this as well; Stevens does date from 1999, and some things have changed significantly since then.)

What I want from a future version of NFS

Years ago I wrote about my then desires for a better version of NFS here and here. Since then I've revised my opinions somewhat and today (prompted by a comment here) I want to talk about it again.

We have two use cases, which for convenience I will call 'NFS v3' and 'Samba'. The NFS v3 use case is traditional Unix filesystem activity between trusted machines, where it is an important feature that the servers trust the client machines to properly identify users (because insisting otherwise means you lose a number of important capabilities). The Samba use case is giving an individual user access to their files without trusting their machine to properly identify users, because their machine is not under our control and security.

Then, what I would like in a future version of NFS is:

• strong 'machine' authentication involving shared secrets via public key cryptography or the like. IP addresses are no longer good enough. We are already kludging our own solution but I would like a better one that is integrated into the protocol.

• per-machine (and per-filesystem) flexible UID and GID mapping. I think that in theory you might be able to do this today with NFS v4 if you had suitable software on the server side (but current Linux software doesn't do it).

  (This would let you handle the 'Samba' case; just give user machines an extremely restricted mapping.)

• optional encryption for the transport stream. It needs to be optional so that it doesn't slow down the 'trusted machines on a trusted network' case.

I used to think that I wanted sub-filesystems to be automatically mounted and accessible. It's okay if this happens but I'm no longer convinced that it's as useful as I thought it would be; in many real cases you do not want how filesystems are associated together to be clearly visible to users because such associations can change.

I don't think you can do machine authentication in NFS v4 as it stands (at least without going fully into strong authentication for the users) but you might be able to get everything else by allowing some machines to 'authenticate' with NFS style 'I trust the client' authentication and then insisting that everyone else support Kerberos authentication.

(Or maybe NFS v4 can actually do machine authentication via Kerberos and then trust the client's UID/GID/name information. I haven't looked at NFS v4 too much for various reasons, including an inaccurate perception that it actively required Kerberos and couldn't be used in the traditional NFS way.)

However I'm not sure that the 'Samba' usage case is very important or interesting for NFS specifically. We do have some people who run their own Unix machines and would like to mount some NFS filesystems but I rather expect that almost all user machines will be Windows or Macs and are not going to have NFS clients (v4 or otherwise). On the other hand, there is one potentially common case of user-run Unix machines and that is cloud computing instances; it would be nice to provide researchers using them with a relatively frictionless way of getting good access to our filesystems.

(Given the bandwidth charges that cloud computing provides like to levy, the researchers might not want to use it. But at least we'd be able to give them an option.)