Wandering Thoughts archives

Notes to myself about progressive JavaScript

Now that I am starting to add various bits of JavaScript to our account request system, I'm being faced with the issues of making things still work gracefully if JavaScript isn't enabled at all. This is not exactly a new issue, especially given that I'm more than a decade late to this particular party, and so I'm writing things down here just to get them fixed in my mind.

(I'm punting on browser-dependent issues by assuming that JQuery will make them go away for any modern browser. As a pragmatic matter I just don't have the time and expertise to do better than this for what are relatively unimportant website augmentations.)

I can split my JavaScript usage into three different classes; things that are augmented by JavaScript, things that are replaced or modified by JavaScript, and things that don't work at all without JavaScript. The cases I need to worry about are the latter two (since augmentation should decay gracefully on its own provided that the augmented feature is not essential).

Let's suppose (not hypothetically) that we have an account request form that asks people for, among other things, their graduate student number. Of course you only have a graduate student number if you're actually a graduate student, so we'd like people to only be prompted for it if they set their status to 'graduate student'. There are at least two approaches to this: we could have that form field hidden in the HTML (or not present at all) and then revealed or created by JavaScript on the fly, or we could have the form field visible in the HTML, hidden on initial page load by JavaScript, and then revealed and hidden again by JS later. If we assume that JavaScript will always run (and run right), the first case is simpler and clearer. But the second case degrades better if the JS doesn't work because the graduate student number field is still there and usable.

(One way we can approach this is to get the actual feature right. Here the feature we really want is 'JavaScript hides the graduate student number if you're not a graduate student', not 'JavaScript reveals the graduate student number if you are'.)

So the rule is: for HTML elements that are replaced or modified by JavaScript, the base element should be present (and visible) on the page with JavaScript hiding or modifying it as needed when (and if) it runs. Or to put it shortly, hiding things degrades better than revealing them.

For things that don't work without JavaScript at all, another example. In the account request system there is a page for handling a bunch of pending requests together where I've added options for 'approve all' and 'reject all'. These options need some visible HTML widgets and they can't work (at least safely) without JavaScript. There are again at least two options: the basic HTML elements could be on the page to start with and made functional by JavaScript, or they could be missing from the base HTML and created by JavaScript when the page is loaded. Again the first option is simpler if we assume that JavaScript always works but the second option degrades better; if JavaScript doesn't run we don't have non-functional HTML elements on the page.

(Among other advantages of the first approach is that it lets you design and lay out the final HTML in one place.)

So the rule is: for things that don't work at all without JavaScript, the supporting HTML shouldn't be on the page at all; it should all be set up on the fly on page load (or whenever) when the JavaScript runs. Or in short, I should never allow non-functional elements to be visible.

(There are actually two options here; you can have the HTML created entirely in JavaScript or you can have it hidden with CSS styling and then revealed by JavaScript. My personal twitch is that I would rather not count on CSS keeping things hidden in all possible browsers, so I prefer the HTML to be entirely absent. However there are probably situations where this doesn't work very well, eg if you are laying out a complex form and you really want to keep all of the HTML in one spot instead of trying to spread it across a base HTML file and various chunks of HTML in your JavaScript.)

Sidebar: the basics of sane modern JavaScript

As far as I can tell, the basics of sane modern JavaScript are basically how a library like JQuery wants you to work:

• put no or almost no JavaScript into your HTML; have it in an external file that the HTML pulls in with appropriate <script> tags.
• do not embed JavaScript stuff inline in various HTML elements as, eg, onchange attributes; instead attach them on the fly when JQuery runs your code after page load finishes. Find the HTML elements to attach to through either id, class, or other selectable attributes.

I have a bit of JavaScript code on some pages but that's just there to set some variables, and the reason it's in the HTML is because I'm generating HTML from Django templates and I want to do variable insertion for things like JSON query URLs (because that makes it easy to switch them around between production and test environments).

(I have no desire to generate my JavaScript source through Django templating.)

Sidebar: an example of what I mean by (pure) augmentation

As part of the account request form, people are asked for the login they want. Obviously they can't get a login that's already taken (or reserved). The web application validates their chosen login when they submit the form (re-serving the form with appropriate error messages if there's a problem), but this is a little bit unfriendly; faster feedback would be nice. So I've augmented the login field with a bit of JavaScript that tries to validate your chosen login on the fly, inserting either an error message or a 'looks good' message into the form as applicable.

This is pure augmentation because it needs no additional HTML elements added to the page and the existing HTML elements remain completely functional even if JavaScript isn't running. JavaScript just adds a bonus feature that makes things friendlier.

What you need for migrating web content

In light of the wiki trap and the question of how to manage content, let's start with a basic question: suppose that you want to migrate from one way of storing and managing web content to another (such as in to or out of a wiki). What things do you need from your current content system?

Based on our experiences here, my answer is that the minimum you need is:

• a list of the important URLs you have now, especially URLs with essentially static content (let's call these 'content pages').

  (Don't assume that this list of URLs is obvious or that of course any random content handling system will give it to you. Neither is true, and if you don't have a list of URLs you get to spider your own site to recover it.)

• each chunk of content in basic straightforward HTML, labeled with the URL it belongs to. When a page is actually composed of multiple chunks of content (for example a blog entry and a series of comments on it) you should get the HTML for each chunk of content separately.

• the metadata for each chunk of content.

  (What exact metadata depends on what metadata your current system collects and maintains. A basic set of HTML pages probably has very little metadata; a blog based site with comments might have a lot.)

With HTML for your content, information about what URLs you need to either recreate or redirect, and the important metadata you can rebuild a basic version of your web area in either another system or with plain HTML pages. How much you need metadata depends on how much use you make of it. A support web area may not really care about things like authorship and publication date while a blog cares a lot, especially once you get to comments.

(I've reluctantly concluded that exporting any edit history is not usually important enough to be included in the minimum list. It's nice to have but in general not having it is not going to break your site, because usually the current page contents are the important stuff.)

The immediate corollary is that if you are picking a system to hold your web content and you care about not getting caught in the wiki trap, you want to make sure that your chosen system can give you all three of these things. Having these things won't make conversion painless, but it makes it much more feasible.

(In practice the question is never whether or not conversion is possible; it's always possible if you work hard enough, because in the limit case you manually crawl your website, copy the HTML of your content, de-mangle it, and so on. The real question is whether a conversion is easy enough to actually be done.)

The wiki trap (that we've fallen into)

A number of years ago we had a not very good support website area that was basically a bunch of HTML pages with very little organization and navigation. In late 2009, when things reached the point where the support site absolutely had to be improved somehow, we made what has turned out to be a bad mistake: we turned it into a wiki. I say that this was a bad mistake because about a year later, we discovered that our wiki software was effectively abandoned. Oh, there's still something by the same name being developed, but it uses a different wikitext format and there's no automatic migration from the old wikitext to the new one. As far as we're concerned, it's different software.

This has left us seriously stuck. It's clear that we have to migrate but it's equally clear that doing so will be a significant amount of work because all of the content is 'locked up' inside the wiki's custom markup (and its file organization). Any means of getting an HTML version of the content will require stripping out what is probably a lot of standard markup and styling that the wiki adds, and we may have to resort to spidering our own site just to get all pages.

(We aren't even thinking about trying to extract the historical record of content modifications. That too is locked up inside the wiki, somehow, but it's not important enough to justify the effort to get it out. So we're going to lose stuff in this migration and no, that does not make me happy.)

In the mean time the wiki software has flaws like bad searching, the content has various issues, and the overall navigation needs to be improved, but doing anything in the current wiki is pretty much a dead end. So we mostly haven't touched it; we make minimal changes to update the content when we change our systems and every so often some especially dedicated person does a little bit more. Deeper issues of structure and wiki features remain completely untouched, partly because modifying the software ourselves is just not going to happen.

(Maybe we could do some of these changes without too much work. But every time we contemplate spending some time learning how to improve the current wiki, the inevitable question is why we don't use that time to start figuring out how to migrate and what to migrate to instead of wasting it on work we'll throw away.)

This is what I'll call the wiki trap: once you put your content into a wiki you're probably stuck with the wiki, for better or worse, because migrating away from a wiki you've adopted is generally quite difficult. A wiki is a one-way ratchet, a place where content checks in but it never checks out.

(Technically you can have a wiki that stores content in HTML form. This wiki would not have this issue, assuming that you can get at the raw content. The other way out is for your wiki to support exporting content to raw HTML. By the way, I'm still ignoring the edit history here.)

I don't have an answer to this situation. In fact this situation makes me very unhappy. I like wikis as a general rule, they give you a bunch of convenient features in one place, I really like simple markup, and there's a lot of appeal to allowing a broad bunch of people edit access to our support documentation so they can improve it. But at the same time I can't deny that putting our content into a wiki has turned out to be very painful experience in a way that could easily happen again with pretty much any other wiki software that we move to. It really looks like we'd be significantly better off with our content in dumb HTML in the filesystem (and assembled into web pages in any number of brute force ways).

(You may be tempted to say that our situation is an exceptional one caused by an unwise choice of wiki software. At one level this is true, but at another level how long is the life of the information that you're putting into your wiki now, and how sure are you that something this could never happen to your wiki software over that lifetime? I'm not sure that I can bet on any wiki software that I want to use to have a fifteen year lifespan, for example, and some of our support documentation is likely to live that long.)