Wandering Thoughts archives

What I would now do differently in a file based blog engine

I recently mentioned in passing that if I was building a file based blog engine today, I would do a number of things differently than they are now. Given that file based blog engines are eternally popular, here's my list of those things.

First and foremost, I would not use file metadata for anything, most especially the file date. Using the date of an entry's file as the entry's date is superficially attractive but in practice it complicates your life quite a bit and it clashes badly with using pretty much any sort of modern version control, none of which will preserve that sort of file metadata.

This means that we need somewhere to put the necessary entry metadata. I would require as little metadata as possible to be in the actual files for entries, although I would allow it to be optionally placed there. Most metadata should instead be in plain text database files that are primarily maintained by the blog engine (but which are human editable). To the extent that metadata goes in the actual files, it should go at the end of the file, not at the start, in order to keep the clutter down; after all, the most important thing about an entry is the text you wrote, so it should be at the start of the file.

(Note that I don't consider the entry's title to be metadata in this sense. Metadata is things like the publication date, the unique ID of this entry, and so on.)

Making an entry public would require an explicit publication step via running a command. Files that haven't been published are only visible to authenticated blog users, which would be the blog's author or authors, and have no metadata (or have fake metadata based on bad ideas, like the file modification time). There are two reasons for this; first, you need some way to view drafts through the blog engine without showing them to everyone, and second, requiring an explicit publication step gives the blog engine a chance to collect and make up the metadata about your new entry.

Blog entry metadata is both very important and hard. Doing a well performing blog engine means that you need to collect this information and put it in a single place, so that you do not have to walk all over the filesystem to work it out. In addition, some of the metadata is annoying for people to come up with when the computer can perfectly well either make it up itself (eg, unique entry identifiers suitable for use in Atom syndication feeds) or use a usually correct default value (eg, the publication date, which is usually going to be 'right when I ran the publication command'). All of this means that the sensible thing to do is to collect it once and record it somewhere (and then provide a way of updating that record if and when necessary).

(Some metadata is still going to have to be built by hand, such as the relationships between entries that you need to do good 'related posts' sections. Such metadata should not be embedded into entries but stored in separate files, so that it's easy to modify by itself.)

While the blog engine's internal database can be in whatever format is most convenient for it, the canonical versions of the information should be in human editable plain text files; the publication command updates these files as necessary. Among other things, this means that you can sensibly keep all of your blog's crucial files under version control. You'll want to commit every time you publish or republish, but you'll want to do that anyways; one way or another you need to capture the changes you've made.

(An entry's unique ID is sufficiently important that I would add it to the entry's file. Among other things, this means that the blog engine can recover from you renaming or moving just the entry file without moving or updating metadata tracking files to go along with it.)

In terms of URL structure, I am relatively convinced that I don't want the entry's file name and directory hierarchy to have much, if anything, to do with the entry's URL; good human usable file names make for at best so-so URLs, as you can see all the time here on WanderingThoughts. This implies that the publication command should normally make up the URL of the entry based on something useful such as the original title of the entry (or perhaps random numbers).

Why blog calendar widgets are bad

I've kind of written about this before, but real blog usability drives home why calendar widgets in blogs are a waste of space for most purposes: as a navigational element, they are at best very weak hooks to get people to explore the rest of your blog. There are two parts of this.

The first is that they give people very little or no information on what they'll get when they click on a link (or even when they look at it and think about whether or not to click). In a typical calendar layout the best you can do is give people the title or titles of entries if they hover over a particular day, and most blog calendar widgets don't bother doing even that.

The second is that navigating through a typical calendar interface is tedious and time consuming. Part of real blog usability is giving people something right away, because if they click a link and don't get something interesting on that page they're quite likely to stop and leave your blog. And if you have a calendar widget that gives people actual entries when they click on things, next look at how much more work it takes to navigate effectively with it, how much work it takes for a visitor to read a third new entry (both in terms of clicks and in terms of mental work to keep track of things).

Let's face it; if calendar widgets are really useful for anyone, it is for the blog's author. So the right place for them is in your administrative interface, not in the view that visitors see.

(I think that there is a role for 'archive' interfaces in general, although not necessarily a very big one, but a calendar widget is a terrible interface for that too.)

Realistic blog usability

Many moons ago, Jacob Nielsen wrote an entry about 'blog usability', which I reacted to at the time. But as I noted back then, Nielsen's advice seemed aimed mostly at what I called 'commercial' blogging, and ever since then I have had simmering in my mind the idea that it misses the true usability issues for typical blogs (then and now). Today I feel like letting things bubble over and finally writing down my thoughts.

To start, let's ask two important questions: how do visitors come to your blog, and what do you want them to do once they've arrived?

Until you get famous, I think that most of your visitors will come to you either from search engines or (these days) from links shared through the 'social web' (Twitter, Facebook, Reddit, etc). In both cases they're mostly going to go to specific entries, not any of your index pages (front page, category pages, etc). My answer for the second question is that once people have arrived at an entry, you want them to stick around and browse further in your blog; ideally you'd like them to become regular readers.

(Back when I first had this thought, search engines were by far the dominant traffic source you could expect. The growth of the social web may have changed that, especially if you're writing the sort of blog entries that attract links. WanderingThoughts still seems to get most of this sort of traffic from search engines, but that's a function of the sort of entries I write.)

These answers have some consequences. First, you really want people to get what they came for, especially if they came from a search engine. At one level this is out of your control, but at another level you want to make the search engines prefer only your most stable and most specific pages, ie the entries themselves. A visitor who is directed to an ever-changing index page is unlikely to find what they're looking for and if that happens, they're unlikely to look any further at your blog; they'll just hit Back (or close the tab) and go somewhere else.

Once a visitor has found what they're looking for on that initial page, what you need next is things that encourage them to explore outwards into the rest of your blog. There are a number of potential hooks here; I think that the most powerful one is a 'related posts' feature, so that your visitor can continue on the topic of the entry that they just finished reading and are (presumably) already interested in.

(A 'greatest hits' feature is attractive, but it sort of requires the visitor to be interested in your writing in general instead of just the specific topic that the entry they landed on is about.)

One way to encourage exploration is to make it less uncertain and risky, specifically by giving people more of an idea what they'll get when they click on a link. Thus, I feel that it's important to have good entry titles and to put them in links on other pages; for example, links to the next and previous entries are (much) more attractive if they show the entry titles than if they're just labeled 'next' and 'previous'.

(As a corollary, category and tag links are less attractive than a 'related entries' set of links that actually shows the titles of those related entries. Even if the actual category page shows more information, people have to do that initial click based on very little but trust.)

This does make me think that the 'many entries on one archive page' model of blog archives may be less insane than it looks, since it lets visitors browse further with essentially zero effort; this is probably especially attractive if you archive things by topic instead of by date. However, to make this really work I think you'd want some JavaScript magic to automatically position the page at the entry that best matches the search terms that search engine visitors used. Otherwise you risk presenting searchers with a large mass of text that isn't really what they were looking for, causing them to bail out on you.

Astute observers will notice that WanderingThoughts falls down on many of these measures. One of the reasons for that is that I had all of these thoughts only after I wrote DWiki, the engine behind WanderingThoughts (and I wrote it for a different purpose, to boot). If I was building a file-based blog engine today, it would look quite different.