X graphics rendering as contrasted to Wayland rendering

March 3, 2024

Recently, Thomas Adam (of fvwm fame) pointed out on the FVWM mailing list (here, also) a difference between X and Wayland that I'd been vaguely aware of before but hadn't actually thought much about. Today I feel like writing it down in my own words for various reasons.

X is a very old protocol (dating from the mid to late 1980s), and one aspect of that is that it contains things that modern graphics protocols don't. From a modern point of view, it isn't wrong to describe X as several protocols in a trenchcoat. Two of the largest such protocols are one for what you could call window management (including event handling) and a second one for graphics rendering. In the original vision of X, clients used the X server as their rendering engine, sending a series of 2D graphics commands to the server to draw things like lines, rectangles, arcs, and text. In the days of 10 Mbit/second local area networks and also slow inter-process communication on your local Unix machine, this was a relatively important part of both X's network transparency story and X's performance in general. We can call this server (side) rendering.

(If you look at the X server drawing APIs, you may notice that they're rather minimal and generally lack features that you'd like to do modern graphics. Some of this was semi-fixed in X protocol extensions, but in general the server side X rendering APIs are rather 1980s.)

However, X clients didn't have to do their rendering in the server. Right from the beginning they could render to a bitmap on the client side and then shove the bitmap over to the server somehow (the exact mechanisms depend on what X extensions are available). Over time, more and more clients started doing more and more client (side) rendering, where they rendered everything under their own control using their own code (well, realistically a library or a stack of them, especially for complex things like rendering fonts). Today, many clients and many common client libraries are entirely or almost entirely using client side rendering, in part to get modern graphics features that people want, and these days clients even do client side (window) decoration (CSD), where they draw 'standard' window buttons themselves.

(This tends to make window buttons not so standard any more, especially across libraries and toolkits.)

As a protocol designed relatively recently, Wayland is not several protocols in a trenchcoat. Instead, the (core) Wayland protocol is only for window management (including event handling), and it has no server side rendering. Wayland clients have to do client side rendering in order to display anything, using whatever libraries they find convenient for this. Of course this 'rendering' may be a series of OpenGL commands that are drawn on to a buffer that's shared with the Wayland server (what is called direct rendering (cf), which is also the common way to do client side rendering in X), but this is in some sense a detail. Wayland clients can simply render to bitmaps and then push those bitmaps to a server, and I believe this is part of how waypipe operates under the covers.

(Since Wayland was more or less targeted at environments with toolkits that already had their own graphics rendering APIs and were already generally doing client side rendering, this wasn't seen as a drawback. My impression is that these non-X graphics APIs were already in common use in many modern clients, since it includes things like Cairo. One reason that people switched to such libraries and their APIs even before Wayland is that the X drawing APIs are, well, very 1980s, and don't have a lot of features that modern graphics programming would like. And you can draw directly to a Wayland buffer if you want to, cf this example.)

One implication of this is that some current X programs are much easier to port (or migrate) to Wayland than others. The more an X program uses server side X rendering, the more it can't simply be re-targeted to Wayland, because it needs a client side library to substitute for the X server side rendering functionality. Generally such programs are either old or were deliberately written to be minimal X clients that didn't depend on toolkits like Gtk or even Cairo.

(Substituting in a stand alone client side drawing library is probably not a small job, since I don't think any of them so far are built to be API compatible with the relevant X APIs. It also means taking on additional dependencies for your program, although my impression is that some basic graphics libraries are essentially standards by now.)


Comments on this page:

By mappu at 2024-03-03 23:57:20:

Substituting in a stand alone client side drawing library is probably not a small job, since I don't think any of them so far are built to be API compatible with the relevant X APIs.

One example of such a library is `xlibe` (https://github.com/waddlesplash/xlibe). This implements the Xlib API entirely in-process, there's no actual X11 server. Effectively, it's a drawing library.

By B.Preston at 2024-03-04 13:24:41:

In the days of 10 Mbit/second local area networks and also slow inter-process communication on your local Unix machine, [server-based rendering] was a relatively important part of both X's network transparency story and X's performance in general.

An aspect you're overlooking is memory usage. X dates back to 1984, when the idea of spending something like 100K for each window was untenable. Thus, till around the late 1990s, graphics systems based around drawing were common—including MS Windows, QNX Photon, and many others. Programs would be told when it was time for them to draw (because previously-invisible portions of their windows became visible), and they'd be given direct access or protocol-mediated access to "the framebuffer". Delays in drawing would often be user-visible. (Colour palettes tended to result in visible artifacts too; younger readers may find the concept of web-safe colours interesting, though Wikipedia lacks any example of how bad things could look when "unsafe" colours were used.)

Now, when we have gigabytes of memory, it's no big deal to have two or three 8-megabyte buffers for each window. Even with double-/triple-buffering, "true colour", and higher resolutions, the space needed for window bitmaps increased much more slowly than system memory. So, "modern" graphics systems are based around buffers instead of drawing. Programs are expected to keep their buffers up to date, and often have no knowledge of whether (or to what extent) they're visible; the graphics server can grab data from those buffers as required, eliminating the flickering.

With 100-megabit networks, and especially gigabit, client-side rendering with X is often faster across these networks than server-side, because most X clients don't handle latency well—waiting for acknowledgement of each drawing command instead of doing it in batches. On a campus LAN the difference is hardly noticeable, but when latency gets above a few milliseconds, one's usually better off having X clients connect to a server on the same machine and using something like VNC to transport the bitmap data.

Written on 03 March 2024.
« Something I don't know: How server core count interacts with RAM latency
An illustration of how much X cares about memory usage »

Page tools: View Source, View Normal.
Search:
Login: Password:

Last modified: Sun Mar 3 22:56:12 2024
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.