2022-10-20
The programming challenge that is a modern browser
Writing and maintaining a modern browser is probably one of the most challenging programming projects you could ask for; it has to deal with inherently hard problems and a large scale.
A modern browser is a relatively or very concurrent and very asynchronous (with all sorts of unpredictable network and input events happening all over the place), and is expected to be a highly responsive system. It faces a hostile environment at multiple levels; not only are attackers trying to exploit it, but various websites themselves may be uncooperative and resource-hogging (either through bugs or because, for example, they have Javascript cryptocurrency miners). Pretty much all browsers now use various forms of strong process isolation for some code, which means that internally a browser is starting to be like a collection of microservices, some of which are distrusted. Browsers are routinely fed bad or actively hostile data, but we demand that they be highly reliable, basically never crashing or failing seriously. A browser must use minimal memory and CPU even under demanding circumstances, and of course not leak resources despite being left running for days or even weeks.
Browsers cover so much ground and are so large that they effectively contain multiple substantial projects as subsystems. You have a Javascript execution system (and now a WebAssembly one too), HTML and CSS parsing, web page DOM rendering (including complex text rendering in a wide variety of languages and fonts), a complex TLS system, a networking system that handles multiple protocols, an asynchronous DNS resolver, image, audio, and video format decoders and players, database and cache layers, and so on. All of these subsystems are complicated by security concerns and many of them interconnect with each other at various levels. Some browsers also have to be cross-platform on top of this.
As if this wasn't enough, browsers are developed on an ongoing basis by relatively large teams. There's a substantial ongoing code and feature churn because things like Javascript, CSS, and TLS keep evolving, new security threats are discovered, and so on. With large teams there's almost certainly regular developer turnover, with new developers coming in who aren't familiar with the project's codebase and its traps. Some browsers accept various levels of 'drive by' contributions from people who don't work full time on the codebase (or at least don't do it all the time, even if they may spend a couple of months working on some feature).
Large code bases generally need abstractions to be tractable, so one way or another a browser's code is likely to contain a large number of internal abstractions. Some will be contained with a subsystem (if all goes well), while others exist at the boundaries between subsystems. Because the concerns about memory usage, CPU usage, concurrency, and responsiveness to asynchronous events are global ones, they may leak through much of the codebase, which adds its own complications.
(For instance, a browser that cares about CPU usage (and network bandwidth) may want to abruptly stop a relatively large number of things the moment you close a tab.)
The challenges of browsers and browser development effectively preclude a number of language technologies. For one prominent example, implementing a browser in a garbage collected language may get you a number of valuable things, but it probably results in too much memory usage to be viable (especially when faced with perverse inputs, which you'll eventually hit).
(None of this is new and it may well all be well known, but today I wanted to write it down for reasons outside the scope of this entry.)