NewFeatures: Recent Entries For range/5-13

New: DWikiText has 'processing notes' (and better quoting)

These are directives that change how DWikiText is interpreted to do things like turn off certain font characters or map a simple-to-type character sequence like '->' to a HTML entity. They are documented in DWikiText so I am not going to repeat myself here.

In the process I added a new and less annoying plain quoting mechanism: ``...''. It looks better in ASCII than it probably does in the font here.

(The code for this was written in August of 2007 or so, but I sat on it because I wasn't entirely sure I liked the feature. Well, nuts to that; time to roll things out and just go.)

New: directories can have an index 'page'

I've decided that sometimes I really want a directory to have an index page, not just a list of the contents. So now I can, with the unimaginatively named 'index' view. It's mostly template based; the normal template uses inject::index to display an __index file. However, the view has several special properties:

• unlike other directory views, it is not inherited by subdirectories.
• it is only listed as an available view in the page tools area if there is a file called __index in the directory.
• if __index is a redirection, the index view will just generate a redirect to the target.

(Thus, the index view could be used to replace the wikiroot configuration directive.)

The index view is valid on directories even without a __index page in the directory. Right now the template just displays a normal directory listing in that case.

New: #pragma search ...

I got tired of the relative links in my blog drafts (written in an entirely different directory than their eventual home) not working. So, introducing #pragma search; this adds additional directories to search for pages when resolving relative links, both WikiWords and explicit [[...]] links. The directories listed in a search pragma must be absolute path names, and are searched only after the hard-coded possibilities have been exhausted (including the alias directory for WikiWords).

You can't have both a #pragma search ... and a #pragma pre in the same page, partly because it doesn't make any sense. (This is the simple way of getting out of handling multiple #pragma directives.)

New: optional disk-based caching

Having run out of other ways to really improve performance, I added a disk-based caching infrastructure to DWiki and then put in two caches.

The real cache is the renderer cache, which stores the results of selected renderers (currently just the wikitext renderers). Via some glue it's also used to store the results of the filesystem walk that's the expensive bit of blog::prevnext.

The Brute Force Cache is for dealing with Slashdotting style situations; it just caches complete requests for N seconds when the system seems to be under load. I also hijacked it as a convenient place to add extra caching for Atom feeds and to force this caching on software that doesn't do conditional GET.

(For more details, see Caching.)

This required a new storage pool class. Like the comment store, it uses a customized and restricted interface to write things (and a new interface to read them). The cache storage pool stores objects, not data blobs, using the cPickle module to make the swap back and forth. (This may be a mistake, but it's fast and easy.)

Since removing files in DWiki makes me nervous, I didn't bother to implement any sort of cache cleaning; you get to do that by hand. The cache has TTLs, and the renderer cache has validation layered on top of the cache object store, but when they detect something invalid they just ignore it. (On the other hand, the cache storage layer does use temporary files and rename(), so in a sense it's already removing files.)

In theory the cache interface is generic, so later I can hook up a memcached setup or something without having to change higher-level code.

New: Text formatting via macros

I (ChrisSiebenmann) found myself with a genuine need for text set in HTML <strike>. Rather than try to invent a formatting setup for this that did not make me cringe, I just punted to the easy solution: macros that do text formatting.

So, DWiki now has grown the new macros:

• ST for the font styles DWiki didn't already have.
• C for character entities, and ShowCharEnts to show the named entities we support.
• because I was there anyways, AB for <abbr>, so I could have those cute inline abbreviation expansion things. Tragically <abbr> is not supported by IE 6 and less, so AB may quietly change to generating <acronym> someday.
• and finally, IMG to generate <img>. Width, height, and alt text is mandatory, and there is a hacky way to also roll title text in too. (Title text is optional.)

Through special black magic, ST, C, and AB can be used in comments (the omission of IMG is deliberate). In theory this lets a commenter cause character set explosions, but in practice a bad commenter can just write UTF-8 directly (UTF-8 is the common and only sane character set choice, so).

Implementing these as macros means that they have some limitations. You can't nest C, AB, or a differently styled ST inside an ST, and currently none of them can be done inside link text ([[....]]).

These macros are a bit of a hack. It's relatively easy to implement bits of HTML this way, but I'm not sure if it's good design overall.

New: Previous and Next links in pages in blog views

The more I have used DWiki for a blog, the more I've realized that I want individual entries to be able to have Previous (entry) and Next (entry) links. At first I resisted because this would require an expensive filesystem walk on even individual page views, but I have now given in and made the blog::prevnext renderer, which will do this if I want.

blog::prevnext generates links directly to the pages, not to /range/N-N/ virtual directories, because I think that works better. (It would be less code the other way, but better links are worth the code.)

New: DWiki can generate Google Sitemaps

I added the ability for DWiki to generate Google Sitemap format XML files, using the view 'sitemap'. The information included is very basic currently: just the URL for each file page, all of them set to priority 0.8 (in the hopes that Google will decide that all of the directories are priority 0.5 and prefer returning file page results).

Google does not say what Content-Type you should return sitemaps in, so I have opted for 'application/xml'.

In the future, something as elaborate as Atom rendering may be done. For now, everything is hardcoded in the sitemap::minurlset renderer.

Updated: now directories are shown too, at priority 0.6. This feature is clearly going to be in flux for a while.

New: https:// now supported in plain text

As part of fixing Atom feeds to not break embedded https:// urls, I decided that we should support plaintext https:// URLs, like say https://bugzilla.redhat.com/.

DWiki should now support non-HTTP URLs much better in general (before, there were a number of problems and issued). You can even include mailto: links if you really want to.

New: Atom feed autodiscovery

There's a standard for autodiscovery of Atom feeds, involving <link rel="alternate" type="application/atom+xml" href="..."> element in your <head>. Now DWiki has a atom::autodisc renderer to create them.

The current code only generates 'recently changed pages' Atom feed links, and so disappears entirely when there isn't one. In theory one can have multiple autodiscoverable feeds (the first is the default, and they get title="..." elements), but I don't quite feel like being that daring just yet.

(I am also not confidant that clients have the UI issues involved sorted out. I'm not sure I have the issues sorted out; for example, should file pages have only the comment feed in the autodiscovery, or should they also have the recent changes for their directory feed in? Which better matches practical user expectations? Can I expect users to be aware of the difference between directory pages and file pages?)