Encryption for stream based protocols versus 'RPC' protocols

May 24, 2023

In my views on protocols and encryption, I called SMTP and IMAP 'stream based protocols' without really explaining what I meant and why this mattered (and why NFS v3 isn't really one, even though it's also transported over TCP). While writing a comment on that entry I came to a realization about this in the context of encryption. The short version is that stream based protocols have context, or equivalently that a specific connection for such a protocol has state, state that's not explicitly specified in each of the messages that are exchanged over the connection (but instead established from the sequence of messages).

When you make a SMTP or IMAP connection, you go through a series of steps to establish what you're doing. In SMTP, there is a sequence of (for example) EHLO, STARTTLS, EHLO, MAIL FROM, RCPT TO, and then DATA and end of DATA. In IMAP, you'll LOGIN and then likely sooner or later SELECT a mailbox and begin reading and manipulating messages in that mailbox. These initial setup operations establish the context for later commands and create a state that your connection is in. What effect various commands has depends on the context and some commands are only valid if the connection is in certain states.

(As a corollary, it's well defined what happens and what you do if a given stream connection breaks. You have to start over from scratch, re-establishing this context.)

Because the context that commands are issued in is a critical part of what they mean, if you add encryption to the protocol this context must be protected in order to insure the integrity of any given connection. In a stream based protocol, you must protect not just the individual commands and their responses; you need to protect the sequence of operations, because removing an operation, repeating an operation, or splicing in another operation from earlier in the stream can completely change the effects and meanings of subsequent commands. Most commonly this is done by simply encrypting the entire stream from beginning to end.

In many RPC based protocols, such as NFS v3, this is at least theoretically not an issue; each RPC operation is supposed to stand on its own (in a protocol sense; clients definitely do have a context that each operation is happening in). If an operation needs some context, such as what directory you're looking up a name in, the RPC request will explicitly carry that context as part of the RPC data (and the RPC reply carries the context of what it's replying to). If the other end doesn't recognize the context or the context is now invalid, the operation will be rejected. In theory this makes it much less critical to protect the entire sequence of RPC operations if you're encrypting things, although you almost certainly still need protection against repeated (replayed) operations.

(RPC operations may not even have a global ordering, the way that all commands and responses in a single connection of a stream protocol do.)

There are a number of interesting RPC protocols, or at least ones that are of interest to me. DNS is a RPC protocol in that you send individual DNS queries and receive replies to them without any particular context being supplied. NFS v3 is definitely an RPC protocol; any context is explicit in NFS RPC requests and replies (although in practice the NFS v3 protocol has protection against repeated requests). Arguably HTTP is an RPC protocol as well, even with mechanisms like Keep-Alive to reuse an existing connection for additional requests.

(HTTP is an interesting case since it's transported over a stream, and you may have to consume and send significant volumes of a stream as part of each individual HTTP request and reply.)

Written on 24 May 2023.
« Some views on protocols and encryption
That people produce HTML with string templates is telling us something »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed May 24 21:23:03 2023
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.