Subdirectories: NewFeatures.
A brief sketch of the Dwiki processing model
The core of DWiki is a template expansion engine and a collection of (text) renderers; DWiki displays pages by figuring out what template to use and then rendering it out.
Renderers generate text based on the current context, such as the page that is being displayed. The most important (and largest) renderer is the wikitext renderer, which takes page content in DWiki's wiki text format and turns it into HTML.
Other renderers create things like the navigation 'breadcrumbs' up at the top of this page and and the page tools and last-modified lines at the bottom. Renderers generally create only the essential pieces of that information; surrounding text is created through template expansion. Renderers are hardcoded parts of DWiki and are thus written in Python.
Templates are text files; they get expanded by the template engine through a recursive process of applying template 'macros' to their text. Template macros can insert other (expanded) templates, insert text taken from context variables, and insert the results of renderers. A typical template might look like:
<html><head><title>${|wikititle|wikiname} :: ${page}</title></head> <body> @{breadcrumbs} <br/> @{wikitext} <hr> #{footer.tmpl} #{site-sig.tmpl} </body> </html>(the actual templates that render this DWiki are somewhat more complicated than that, but this shows the flavour.)
DWiki produces all pages this way. Displaying different types of pages (regular pages versus directories) and different views of the same page (such as the history view) is done by selecting a different starting template; the template (presumably) uses different renderers that the normal view.
Errors are also rendered using templates (if an appropriate template exists). This allows some error pages to reuse renderers as appropriate; for example, the no-such-page error template includes breadcrumbs just as regular pages do, as you can see at NoSuchPage.
Wart: the view source display is not done by a template: it just barfs the content out straight as plain text. One current limitation of renderers and templates is that they can't control the content-type, which is set in the HTML view core.
Wart: the mapping of view + file attributes to templates is currently hard-coded.
The frontend versus the core
DWiki is divided into two components: the front end and the core. The front end receives raw HTTP requests, figures out if they are proper requests, and then passes them to the core to go through the core's processing. If the front end can detect that a HTTP request is not something that the core can handle, it rejects it immediately with a terse error.
Similarly, if the core encounters a processing error it throws an exception up to the front end, which logs it and generates another terse error.
It is the front end that can optionally serve static files; the core is not involved in that process.
DWiki features
DWiki's job is to be a good way to display version controlled wiki-text pages that you write in a real editor.
The important DWiki features:
- simple but reasonably powerful text rendering (based on WikiText).
- natural support for arbitrarily-named links: you don't have to follow some magic page naming standard that doesn't fit well with the natural names for things.
- pages are normal, simple files, and you edit them directly in Unix.
- support for putting pages in RCS, with strong disincentives to hand-edit files without checking them out (they stop displaying).
- directories can display like changelogs: pages inline, most recent first.
- can generate Atom syndication feeds for recently changed things.
The inevitable feature list:
In no particular order:
- simple WikiText-like text rendering. (Chris wrote pages in GNU Emacs and relentlessly smushed anything that got in the way of how GNU Emacs wanted to autoformat things.)
- The text rendering choices are designed to make it easy to write about Unix systems.
- full support for directly editing wiki pages.
- does not force a flat page namespace; uses straightforward Unix files and directories to organize the DWiki page space. (Thereby keeping the Unix view of DWiki's pages simple.)
- supports a blog-like view of a directory that inlines pages there, most recent first.
- in-filesystem page redirects make it trivial to support plurals, moved/renamed pages, etc.
- text-based page templates control how all pages appear, making it easy to control various bits of a DWiki's appearance.
- pages can be put in RCS for version control and multi-person editing access. RCS files can live in either the page directory hierarchy (for simplicity) or another parallel directory tree (for neatness).
- forces people not to edit RCS-controlled files without locking them by refusing to display inconsistent unlocked files.
- generates Atom syndication feeds for recently changed pages and recent comments, for the entire DWiki or any subtree thereof.
- written in Python.
- simple-ish yet powerful enough (I hope) user authentication system, with an equally simple yet powerful way of restricting who can read DWiki pages.
- supports the option of letting people (possibly including the world) comment on some or all of the pages.
- takes some pride in properly generating and handling Last-Modified: and ETag: headers in HTTP responses.
- wikitext to HTML generates fully HTML 4.01 Transitional compliant HTML provided only that you don't jump multiple indent levels in at once in lists (thus Formatting doesn't validate).
- can run as a CGI-BIN or standalone, and support for additional environments (SCGI, WSGI, whatever) should be easy to add if it is needed. Disclaimer: standalone does not use a production-quality webserver implementation; it uses Python's
BaseHTTPServerwith a hack to use threading.Missing DWiki features
Also in no particular order:
- you can't edit DWiki pages from the web, but see WhyNotWebEditing.
- no user authentication.
- therefor, no access restrictions on who can read what.
- searching is primitive at best.
A necessary acknowledgement:
A number of DWiki's features and design decisions are shamelessly inspired by C.J. Silverio's as yet (22 May 2005) unfinished Snippy. Note that Snippy is much more powerful than DWiki probably ever will be, plus if it had been finished when I was writing DWiki I probably wouldn't have.
Page Names That DWiki Won't Serve
There are some paths and page names that DWiki categorically refuses to serve, even if they seem to resolve to real files. Because they're enforced by both low-level code and high-level code, they apply to DWiki pages, static files being served by DWiki, and even templates. (Technically they apply to comments too, but comments can't generate file names that violate these rules.)
What gets rejected:
Any path that includes a path component that starts with a
., ends with,vor a~, or isRCS.Any non-relative path that includes
..,., or a sequence//; usually this might appear in the URL of an incoming request. (Incoming requests are not supposed to include things like that. But ChrisSiebenmann declines to believe that everyone sending DWiki requests is going to do what they're supposed to.)DWiki will reject REDIRECT files that either have too many '..' entries (so that they are trying to escape the root of the page directory) or that fail these checks after they've potentially been converted from relative path names to absolute inside-DWiki paths.
When DWiki rejects bad paths, generally it says that there is no page by that name. Sometimes it rejects the request entirely in huge flames.
Redirection Files
Files in the page directory can create HTTP redirections, making it trivial to support plurals, moved/renamed pages, and so on. There are two ways of doing it:
REDIRECTcontent and symbolic links.If a file starts with a line that says '
REDIRECT somewhere', and does not have more than a few lines of content, DWiki considers it a redirection. The somewhere is basically interpreted as if it was appearing in a[[....]], so it can be:
- redirection to another DWiki page.
- redirection to an external web site, written as
http://....- redirection to an absolute URL on this web site, written as
<...>These files are generically called REDIRECT files.
A symbolic link is only considered a redirect if DWiki can 'resolve' it into an existing page. To resolve the symbolic link redirect, DWiki tries to interpret the symbolic link's value as if it was appearing in a
[[...]]as a DWiki relative page name.If the symbolic link doesn't resolve this way, DWiki treats the whole thing as an ordinary page; this keeps 'ordinary' uses of symlinks intact in most cases, including when the symlinks point to something outside the DWiki page directory.
Redirects to http:// links or absolute URL links are a convenient way of creating WikiWord abbreviations to external things for local use. Make an appropriate REDIRECT file, stick it in your Aliases area, and now every page in the DWiki can say GoogleSearch or something and get a link, bam.
(WikiWord redirection rewriting means that in many cases the generated link will even point to the real target instead of the REDIRECT file, as you can see here.)
Hello, I am a DWiki
DWiki stands for 'Dinky Wiki', which is what ChrisSiebenmann calls this for lack of a better name. As far as Chris knows on a casual scan, 'DWiki' is not being used as the name of any other Wiki software.
I (ChrisSiebenmann) wrote DWiki to have a good environment for writing documentation about how we do system administration work on our Unix systems. You can see a bigger list of Features if you want.
DWiki's design choices are very slanted towards making it as easy as possible for us to write, revise, and maintain system documentation. (Because on past evidence, it if isn't dirt easy we just aren't going to do it.)
I like WikiText, because it is a simple, low-effort way to write documentation that comes out looking decent. But I don't like the wiki interface for writing it: web browsers make crappy editing environments and limit the sort of changes you can easily do. So I wrote DWiki to have a good system for displaying wikitext that we'd write through other means.
Wiki or CMS?
Some people say (strongly) that DWiki is not a Wiki because people (currently) can't edit DWiki pages over the web; they argue that instead it should be called a CMS (a (web) Content Management System).
I'm willing to accept this argument in theory, but ...
I like punchy, usefully descriptive names. My personal opinion is that 'wiki' is one: calling DWiki a wiki tells you a great deal about it, while calling it a CMS tells you almost nothing.
So while I may be willing to accept the argument, DWiki stays being called a wiki (and keeps the 'Wiki' in the name).
I maintain web-based editing makes little sense for us, and wrote a bunch about why in WhyNotWebEditing.
And one can always have a big debate over at the granddaddy Wiki, at http://c2.com/cgi/wiki?WikiPrinciples.
(One comment.)
Why DWiki doesn't have web page editing
One of the signature Wiki features is that people edit pages over the web, often anyone and without restrictions (as the original Wiki was/is). It's said that this is a defining trait of Wikis, and that without it what you have isn't really a wiki.
DWiki has no from-web editing of pages. There are several reasons why.
Interface power
A web browser's form input text boxes are a totally crappy editing environment compared to what I have on a Unix system. Yeah, sure, I could require Javascript and load a huge editing library and maybe get somewhere, but a) I browse with Javascript off and b) am I going to get half as good as GNU Emacs or vi or sam? (I don't think so.)
So I want the primary way of editing DWiki pages to be from Unix, through the filesystem, with real editors. (And it is.)
Global edit doesn't make sense for us
The principles of global edit permissions leading to the world help write your pages simply don't make sense for us. DWiki's goal is to let us easily document how our Unix systems work. We're the only people who can write most of that documentation; outsiders can at best add side commentary.
This would be different if we were interested in running a Wiki on system administration best practices or the like. But we're not; we're just documenting our systems. We let other people read it so that they can learn from anything interesting we do (and that's primarily aimed at other people at the University of Toronto).
It's a drain we can't afford
Anything that allows semi-public writing on the Internet requires tending. Wikis are no exception to this rule.
Like many places, we are historically very bad at creating documentation. The more effortless I can make the process, the better the odds that we will actually write documentation.
Keeping DWiki running is part of the overall process; the less effort this takes, the better, especially if we aren't actively writing documentation at the time. Thus, I don't want DWiki to take up any time when we're not actively writing things with it.
If DWiki allowed web writing from anything except a small set of people, we would have to tend it. It is simpler and less risky to avoid that, especially given that we can't expect significant contributions from outsiders.
Skipping hard design problems
Eliminating web-based editing immediately kills the need to tackle a bunch of hard problems, because Unix handles them for me. Particularly, I don't need to authenticate people or do access control, provided I'm willing to let everyone read (I am, so far).
Access control, authentication, and registering people is not an easy area. It's also one where failures and program bugs can have severe consequences. Not having to worry about that means that DWiki is faster to write, smaller, and safer.
I also don't have to worry about random outsiders writing pages that make extensive use of expensive DWiki features, or writing things in pages that cause rendering errors.
But web editing can be done from anywhere!
Pragmatically, the odds of us wanting to edit our systems documentation from anywhere that we can't just run
sshto log in to our servers is fairly low. This is especially the case given that there are Java SSH applets, so that any browser that runs Java can let us log in to our servers.DWiki is aimed at the low-hanging fruit of the 90% or 80% or so solution. (I maintain that any wiki is, partly because the text rendering is deliberately simplified.)
The future: maybe limited web editing
The most likely web editing feature for DWiki to pick up is to let web people write comments on pages but not edit the pages themselves. This would let outsiders give us feedback and commentary without running the risks of scribbling over valuable page content.
This would still require me to either write an authentication system or live with the likelyhood of comment spammers showing up to yammer madly. Plus some of the above worries.
(The clever person will notice that some of this future has arrived. DWiki now has an authentication system and comments, although both will be improved in the future.)
(2 comments.)
The code structure:
There's three chunks of code: the HTTP layer, the HTML view core, and the model.
The model deals with view-independant wiki level things, primarily retrieving raw pages and templates. (To do this it calls on a storage pools managed by a storage layer; the storage layer handles much of the RCS magic.)
The DWiki HTML view core gets a request context and is responsible for returning a response, whether that be rendered page content, redirections, or (rendered) errors. Renderers and template expansion are part of the HTML view core.
The HTTP layer is responsible for generating the request context, sending the response (including conditional GETs and other fun), and giving the HTML view core ways of generating proper URLs for given wiki pages.
The split between HTML view core and HTTP layer exists because the HTML view core is agnostic about how it is connected to the web, while the HTTP layer is intimiately tied to CGI-BIN versus Python BaseHTTPServer versus etc. So the HTML view core has everything that is web server independant.
FIXME: investigate this Python SCGI thing I've heard bits about. Or is that WSGI? See http://www.python.org/peps/pep-0333.html and http://wiki.python.org/moin/WSGIImplementations. Unfortunately on a preliminary look it seems I might as well write to CGI-BIN to start with.
I think that this is almost but not quite Model-View-Controller, but then I don't understand how MVC works (especially on the web).
To look at:
- http://java.sun.com/blueprints/patterns/MVC-detailed.html
- http://www.phpwact.org/pattern/model_view_controller
This suggests I am almost MVC except that my Controller is smeared over View code and that I have split the View into two pieces: the HTTP layer and the HTML view core.