Wandering Thoughts archives

Today's learning experience with CSS: don't be indirect

This is today's learning experience and I will preface it by saying that I am probably doing things wrong and in not the right CSS way. I will present this as a story.

Once upon a time, you write a wikitext to HTML converter and with it some associated CSS. Your wikitext has tables and the tables should be styled in a certain way, so you wrap the entire generated wikitext in a <div class="wikitext"> and write a CSS rule:

.wikitext td { border: 1px; border-style: solid; padding: .3em; }

These tables come out with a nice 1 pixel solid border the way you wanted and also the right padding around everything to look nice.

Your wiki also has some tables that it generates outside of the wikitext. They have HTML like <table class="blogtitles"> and CSS to style them the way you want:

.blogtitles td { padding-bottom: .5em; vertical-align: top; }

.blogtitles td + td { padding-left: 0.5em; }

These tables also come out with the right padding and no border, the way you want them to.

Then much, much later you decide that you want to embed a blogtitles table in the generated wikitext, wrapped in that great big wikitext <div>. You render the whole thing and lo, your blogtitles table comes out looking horrible. For a start, it has borders.

Well, of course it has borders. You said to give it borders: 'every <td> inside a wikitext <div> should have borders' says your CSS, and right there is a (blogtitles) <td> inside a wikitext <div>. Similarly your blogtitles table has all sorts of padding it 'inherited' from (general) wikitext tables. The results of combining the blogtitles CSS with the wikitext tables CSS is probably nothing like what you wanted (and may not look very good).

Your problem (ie, my problem) is that you were indirect when you did not want to be. 'Any <td> inside my <div>' is an indirect way of specifying 'wikitext tables', and as an indirect way it runs the danger of being too general. Which is what happened here. Blogtitles tables are conceptually a completely separate thing and should be styled independently from your regular wikitext tables, but they are being swept up in your dragnet.

The right solution, at least in generated HTML, is to be direct. Generate your wikitext <tables> with an an actual class (eg <table class="wikitable">) and then write CSS on that. The CSS doesn't even have to change much. In short, say what you actually mean. You didn't really want to style every <td> inside your wikitext; you wanted to style your wikitables. So you should say this directly (in CSS and in classes) and save yourself a certain amount of hassle and annoyance.

(There is probably a really clever way to fix this in CSS that I don't know because I'm mostly CSS-ignorant. Note that I don't consider carefully trying to undo the wikitext table settings to be a clever way.)

The ice is thinner for HTML that isn't automatically generated, because putting classes on things is somewhat more annoying there (especially if you may have a lot of them). I don't pretend to have a nice answer there.

What makes DWiki and other dynamic file based blog engines slow

In yesterday's entry I mentioned that DWiki (the software behind this blog) is pretty much a worst case for a blog engine as far as speed goes. Today I feel like talking about what makes DWiki slow, and by extension the things that can slow down any dynamic file based blog engine. Part of why is so that you (if you are considering writing such a thing) can avoid the mistakes that I made.

(Some of the slowness is because chunks of DWiki's code are not exactly the best that they could be, but the issues there are generally dwarfed by the general ones I'm about to discuss.)

For basic background, DWiki is about as pure a dynamic file based blog engine as you could ask for; conceptually it is purely a bunch of views of a filesystem hierarchy (actually of two of them). Each entry and each comment is stored in a separate file in a directory hierarchy (entries are files in category subdirectories and comments are files in a per-entry subdirectory that is itself in a mirror of the entry's regular hierarchy). Entries (and comments) are written and stored in DWiki's wikitext dialect, not HTML, and the time of an entry (or a comment) is the modification time of its file.

This gives DWiki two main slow points. The most obvious one is converting DWikiText to HTML. At the level of a single entry it isn't a terribly bad process, taking about 6 milliseconds to render yesterday's entry (and then about 4 milliseconds to render the sidebar text, which is also wikitext in a file). But at the level of the blog front page this adds up fast; ten entries is already over 60 milliseconds (although per-entry rendering varies by a few milliseconds depending on what's in them). Still, 60 milliseconds is not a terrible killer.

(In retrospect, one of the reasons to use Markdown or some other popular wikitext format is that other people may well write fast HTML converters for you. With a private wikitext, you're on your own.)

The less obvious but much larger slow point is that DWiki has to walk the filesystem any time it needs to know the relationship between entries, or just to find them all. The obvious case is the blog's front page, which needs to find the N most recent entries; in a file based engine like DWiki you do this by walking the filesystem to find all the entry files, stat()ing them to find their timestamp, sorting the list, and taking the top N. More subtly, DWiki also needs to do this walk when displaying individual entries in order to figure out what the next and previous entries are so that it can generate links to them. And if you want to display some sort of calendar of what days or weeks or months have entries? Again you need a walk.

(Comments are usually less of a problem because the filesystem walks to find them are smaller and more focused. The exception is if you do something crazy like 'show N most recent comments'.)

This filesystem walk is not a big issue for a small blog (which will have a modest number of files). But when your blog gets more and more entries, well, things scale up and slow down. Rendering the front page of WanderingThoughts without any caches currently takes 3,299 lstat()s and scans 18 directories; rendering yesterday's entry takes 3,207 lstat()s and scans 13 directories. This takes a while even if everything is in the kernel's caches.

(You can optimize the walking code as much as you want but you still have to stat() every file no matter what you do. For scale, a raw filesystem walk over all WanderingThoughts entries currently takes about 200 milliseconds with hot kernel caches (in Python, but ls and find take similar amounts of time).)

The way around these problems is to cache or pregenerate this information, which is why if I was doing a file based blog design again there would be an explicit 'publish entry' step (among other changes).

(DWiki is as weirdly limited as it is because its initial design was to run purely read only, with no write access to anything. Comments and on-disk caching still haven't fundamentally changed that attitude.)

Sidebar: two other DWiki performance-related design mistakes

DWikiText allows bare words (in the usual WikiWord format) to be links if and only if the target of the link exists. This turns out to be a bad idea if you want to cache the rendered HTML, because suddenly changes elsewhere in the filesystem (not just changes to the page itself) can invalidate the HTML; a file appearing or disappearing can create or remove a WikiWord link. This adds a couple of extra lstat()s every time DWiki loads a cached HTML rendering.

(This is not just a performance issue. It means that you can't have a simple model of 'compile the HTML of an entry when it's published and you're done'; you have to worry that publishing a new entry will need an old entry to suddenly be regenerated. The headaches are just not worth it; use a wikitext that requires explicit markup for links and then makes them always be links, whether or not the target exists.)

DWiki has an authentication and permission system that controls things like who can see or comment on an entry. Cleverly I made two terrible decisions when designing it; permissions are embedded in the DWikiText markup and permissions can be per file not just per directory hierarchy. In short, DWiki kind of has to render each file to find out if it can render each file. This is saved only by the fact that generally you're going to render a file anyways any time you need to check its permissions (if it's accessible), but if I was doing it again I would not do this; it could be pretty bad if there were a lot of access-restricted pages.

(DWiki caches this permission information along with the rendered HTML, which helps. The actual code model for doing this is in retrospect kind of terrible, partly because it evolved in multiple steps and was never refactored to be sane.)

Dynamic web sites and blogs need not be slow, illustrated

There is a common myth or perception that dynamic websites and especially dynamic blogs are slow. I've tilted at this particular windmill before, but today I feel like providing some actual numbers for DWiki (partly because I just finished some performance tuneups). DWiki can run either as a CGI (under light load) or as a SCGI server (under heavier load). The interesting load test is for the SCGI server version, complete with its Rube Goldberg hookup. So let me give you some numbers for that.

The full SCGI stack, including the the C CGI program that talks to the SCGI server, can serve this blog's front page in roughly 50 milliseconds a request and a random entry's page in 25 milliseconds a request. Actually those numbers aren't realistic because I turned off two layers of caching in order to slow things down. Under a Slashdot-style load directed primarily at a small number of URLs, those times would drop to 7 milliseconds for the front page and 6 milliseconds for the random entry.

(That time includes starting a C program for every request and having it communicate with the SCGI server. Today's moral is apparently that starting small C programs is really fast on a multiprocessor system, although this is a bit artificial since I haven't tried to run a bunch of them at once as would be happening in a real load situation. The other moral for today is that performance numbers really depend on your assumptions; all of mine here are 'single request at a time'.)

Giving you SCGI numbers is actually a bit misleading, because this blog spends most of its time running as a CGI. So what are the numbers like there? Not as impressive, I'm afraid; DWiki averages 250 milliseconds (ie a quarter of a second) to serve the front page and 215 milliseconds to serve the random entry. Most of that time is the overhead of starting a rather substantial Python program as a CGI. I will note that sane people do not do that. While a quarter of a second is not great response time, I also feel that it's not particularly terrible.

(Sane people would also use a more efficient SCGI gateway than my CGI, and maybe a more efficient SCGI server implementation than DWiki's.)

What these timings show is two things. The first is that running a dynamic website in the right way is important; running DWiki as a CGI costs about a fifth of a second. The second is the vast importance of caches, because even though I turned off two layers of caching there's a third layer that's vital for decent performance. Running as a SCGI server but with no caches at all, DWiki averages 320 milliseconds per request for the front page and 245 milliseconds for a random entry. A completely uncached DWiki request served via a CGI would take over half a second (and use a lot of CPU and do a lot of disk IO).

I'm not going to claim that getting DWiki fast was easy and I'm not going to claim that DWiki was fast from the start. Rather the opposite; many years ago, DWiki started out life as the uncached, CGI-only version that would have rather bad performance today. It took many iterations of both adding caching layers and tuning performance to get it up to its current full speed under load (and I still somewhat regret that it takes so much caching to go fast). On the other hand, DWiki is really a worst case for a blogging engine at this point; it has so many core points of inefficiency and waste that a better blogging engine would avoid. If DWiki can be made to go fast despite this, my view is that anything can. Slow dynamic blog engines are slow because they are slow, not because they are dynamic.

(To put DWiki's problems one way, it turns out that filesystem hierarchies make poor substitutes for a real database once things scale up.)

Sidebar: The hardware involved

All of these timings are still misleading because I haven't told you what hardware all of this is running on. All of these timings are on a 2007 or so vintage Dell SC 1430 with a single Xeon 5000 (3 GHz) and one GB of memory. And it's an active multi-user machine that spends much of its time running relatively heavyweight spam filtering processes and so on (although my timing runs were done at a relatively quiet time).

One of my dreams is to bring up an instance of DWiki and this blog on a small virtual machine (512 MB RAM, say) to see how fast I can get it to go in that sort of minimal environment and how I'd have to configure things. But, sadly, too many potential projects, too little time.

A fundamental problem with the trackback protocol

In vague theory one answer to my problem with the modern web hiding discussions is for the modern web to support the Trackback protocol. Except that the Trackback protocol is (as far as I can tell) basically dead at this point, killed by any number of things including a lot of spam.

When I was thinking about this recently, I realized a fundamental problem with the whole Trackback protocol (besides this issue) and why it was almost immediately abused. To put it simply, the problem with trackbacks is that they are not a means for telling you when someone's linked to your web page; we already have a perfectly good mechanism for that in the form of Referer headers. Instead trackbacks were in practice a protocol for causing links to appear on people's web pages. As the latter, they were always destined to be basically an engraved invitation to spammers.

(Some people will say that Referer only works if readers actually follow the links from the other page to yours. The simple answer is for the blog software or the author to deliberately do that (perhaps several times and/or with a specific User-Agent) simply to get the information into your logs.)

Now I will go flying off on wild speculation. As I understand it, not all web hosting firms give you easy access to things like Referer logs (I have vague memories that cheap low-end web hosts are especially bad about this). In the face of such a hosting environment, it's probably an easier hack to add an entire protocol (which can be implemented in your PHP code by you) than to try to argue your host into giving you log access. And it's more likely to work reliably in that if your site is working at all (and allows you to actually write stuff), it's probably going to be recording trackbacks.

(This line of thought makes me wonder if anything else has been created to get around the limitations of cheap web hosting. Arguably PHP itself caught on because it was easy to host.)