HTTP has become the default, universal communication protocol

July 19, 2023

Back when I wrote about how the mere 'presence' of a URL on a web server wasn't a good signal, I casually mentioned that you were more likely to have everything answered with a HTTP 200 status on things that weren't really 'web servers' as such, but which were just using HTTP. You might ask why you'd use HTTP if you weren't a web server, and the answer is straightforward and widely known: HTTP has become the de facto default communication protocol. Today, if you need to create a system where you pull some information from something or push some information to something, you're most likely to use HTTP for this purpose. In the process, the software may be coded in such a way that it provides a default answer to nearly everything.

In my view, the reason that people see HTTP as the default communication protocol and why so many things use it this way is simple; it works well enough and it's there. Because web servers have become common, pretty much every modern programming environment will have HTTP clients and simple servers in their more or less standard libraries. Some environments (like Go and Python) directly support them in the standard library; others, such as Rust, defer these to third party packages but have good implements and make it easy to use them. There's also wide agreement on and support for a way to transport data over HTTP, in the form of JSON; again, most everything you want to use supports this relatively easily. The conceptual model of HTTP is also simple, in that you can basically view it as an RPC system.

(An additional bonus of using HTTP is that you generally get a bunch of tools and features more or less for free, although people may start asking you for extra things in your software once they start taking advantage of this. For example, people who run your 'web server' behind a reverse proxy often start asking you for some basic URL mapping features.)

You don't have to use HTTP and in a number of cases you're better off doing something else. But it's pretty certain that doing something else will require you to write more code, and you'll be cut off from the ecology of tools that support HTTP and can act as ad-hoc clients and servers for testing, diagnosis, and so on. Similarly, you don't have to use JSON as your data encoding over HTTP, but if you don't you're probably writing more code and putting yourself through a harder time (cf).

(Using HTTP will in theory also give you a number of best practices, although how applicable they are to things that are only using HTTP as basically a transport protocol is an open question. There are ideas you can borrow from 'REST', though.)

None of this is new or a novel observation, of course. I don't know when HTTP reached this state of being the default client/server communication method that got used, but it's certainly been quite a while.

Comments on this page:

By orev at 2023-07-20 14:08:48:

I believe that one of the main drivers of using HTTP for everything is that firewalls don’t block it. In the old days, where every app had its own network port, it became possible for the security team to control access to everything by blocking ports. Since HTTP was so useful for web sites, it had to be left open. Then everyone started using it because it allowed circumvention of all the port blocking. Now we need to implement HTTP firewalls that inspect URLs to try to block apps. It’s a slow moving arms race between businesses and security teams.

By cks at 2023-07-20 15:26:26:

I don't think that firewalls are a large factor in the use of HTTP as a transport mechanism. Many of the things that use HTTP this way don't normally run on port 80 (or port 443 when they use TLS), and they often are only used inside a network perimiter. I'm thinking here of software like the Prometheus host agent or its Blackbox probe agent, both of which use HTTP but only sort of incidentally, and which you definitely don't want to expose to the world.

In a past life (and even for some software today, such as ClamAV), these things would have used a custom protocol of some sort instead of HTTP. The observable effects would be about the same.

It’s certainly my go-to.

Another aspect is that HTTPS takes care of encryption for you, it’s transparent to most devs.

Written on 19 July 2023.
« Non-uniform CPU hyperthreading is here and can cause fun issues
Command hashing in Unix shells is probably no longer worth it »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Jul 19 22:14:48 2023
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.