2023-08-07
Good RPC systems versus basic 'RPC systems'
In my entry on how HTTP has become the default, universal communication protocol, I mentioned that HTTP's conceptual model was simple enough that it was easy to view it (plus JSON) as an RPC (Remote Procedure Call) system. I saw some reactions that took issue with this (eg comments here), because HTTP (plus JSON) lacks a lot of features of real RPC systems. This is true, but I maintain that it's incomplete, because there's a difference between a good RPC system and something that people press into service to do RPC with.
Full scale RPC systems have a bunch of features beyond the RPC basics of 'request that <thing> be done and get a result'. Especially they generally have introspection and metadata related operations, where you can ask what RPC services exist, what operations they support, and perhaps what arguments they take and what they return. Often they have (or will eventually grow) some sort of support for versioning. Although it's usually described as a message bus instead of an RPC system, Linux's D-Bus is a good example of this sort of full scale RPC system (including features like service registration).
(Large scale RPC systems may or may not have explicit schemas that exist outside of the source code, but generally the idea is there. Historically, some large RPC systems have tried to generate both client and server interface code from schemas, and people have sometimes not felt happy with the end result.)
These RPC system features haven't been added because the programmers involved thought they were neat. Full scale RPC systems are designed with these features (or have them added) because these features are increasingly useful when you operate RPC systems at scale, both how big your systems are now and how long you'll operate them. Sooner or later you really want ways to find out what versions of what services are registered and active, and introspection tools help supplement never up to date documentation (or reading the source) when you have to interact with someone else's RPC endpoint (or provide a new endpoint for a service where you need to interact with existing callers).
However, programmers don't need these features to do basic RPC
things. What programmers often start out wanting (and building) is
an interface that looks like 'res, err := MyRPC(some-name).MyCall(...)'.
Maybe there's a connection pool and so on behind the scenes in the
library, but the programmers using this system don't have to care.
And you can easily and naturally use HTTP (with JSON payloads) to
implement this sort of basic RPC system. Your 'some-name' is an
URL, your MyCall() packs up everything in a JSON payload and returns
results usually generated from a JSON reply, and so on. On the
server side, your RPC handling is equally straightforward; you
attach handlers to URLs, extract JSON, do operations, create reply
JSON, and so on. Since HTTP has become so universal, libraries and
packages for doing this are widely available, making such a basic
RPC system quite straightforward to code up on top of them. Plus,
you can test and even use this basic RPC system with readily
tools like 'curl' (for example, using curl to query your
metrics system).
(If you need authentication you may need to do some additional work, but this sort of thing is often used for basic internal services.)
It's not particularly easy or straightforward to make a HTTP based system into a good RPC system. But often you can get away with a basic HTTP based 'RPC' system for a surprisingly long time, and it may be the best or easiest option when you're just starting out.
(The history of programming has any number of things that were built to be good general RPC systems, but didn't catch on well enough to survive and prosper. See, for example, this list in the Wikipedia page on RPC; some of these are still alive and in active use, but none of them have achieved the kind of universality that HTTP plus JSON has.)