Wandering Thoughts archives

A retrospective on my one Django web application

Looking back, it's been almost four years since I started writing our first and so far only Django web application, a system for automating much of our new Unix account requests. Since retrospectives are relatively uncommon, today I want to do one for what has been a really faithful web app.

I suppose that's a good summary of the whole situation: after I wrote it, the application has been trouble-free. Over the years I've done a few tuneups to make it a bit more convenient to use for some tasks and to reduce the possibility of accidents while people use it (and there was one interesting bug), but that's it as far as code revisions have gone. This does means that I haven't had to face things like schema migrations; the schema we started with is the schema we still have, and it'll probably last all the way to end of life with this application (which is not going to be any time soon, since we'll probably always need something to do this job).

(Okay, I'll admit it; we have little enough data stored in the system that I might not even try a schema migration but instead just do a database dump, edit the dump a bit, and load it into a new database.)

I'm still pretty happy with our choice of Django, SQLite as the underlying database, and mod_wsgi for deployment. All of them work and I don't see any obviously better alternative, either then or now. Django does a bunch of magic with database models, but I'm a pragmatist; the magic works.

My one real regret about the structure of the web application is that it has (gasp, shock) no tests at all. The core reason for this is that learning Django testing at the same time that I was learning Django was just too much work, and of course there has never been either the time, the energy, or the clear need to go back afterwards to figure out how to do it and add tests. This doesn't make much of a difference for development since there isn't much development, but it does make a difference for upgrading Django itself. And frankly that's one area I've been weak on; we're behind in both our Django versions and jQuery and part of that is because testing new versions is not drop in and go but instead manual.

We're behind on Django versions primarily because we rolled our own; when we built the application initially, the version of Django packaged on the Ubuntu version we were using wasn't advanced enough. These days I would use the Ubuntu 14.04 Django version and be done with it for four years or so (ie I'd be counting on the Ubuntu people to feed me non-breaking security updates that we could install without any particular testing). I don't think we're actively at risk, but it's something I do worry about.

What I don't like about Django boils down to two things, both relatively minor. First, with SQLite we've found that Django's internal 'id' primary key field can reuse ID numbers if and when old records are deleted. We use and expose these ID numbers in a few places (for example, in audit records) and it would be slightly nicer if they didn't get reused. The most annoying (and potentially dangerous) place is that they appear in the URLs exposed to administrative users, which means that an old URL can go to a completely different account request in six months instead of just being invalid.

(These unguarded URLs are not exposed to non-administrative users and operations on them do have specific security checks, so you can only touch entries you have authorization for.)

Second, I have historically found it hard to find release announcements and changes for new Django versions, which I have to do because I need to read over them as we update from version to version. If I closely followed Django and updated on every version change I think I'd be fine, but as it is now I wind up ignoring Django version shifts for months at a time and then have to grubbing through the site to find all of the release announcements for every version between where we are and where the state of the art is.

(In fact I just went through the Django website and couldn't immediately locate even a 'what is new in the latest version' page, much less ones for previous releases. And I know the information is somewhere, because I've found it before.)

PS: that we have only one Django web app is not because we've grown disenchanted with it, it's just that we haven't had a combination of time and need to write any more (and we do have a second one somewhere quite far down the priority list). Well, for me to write any more; my co-workers don't do Django so far.

Update: It turns out that Django release notes are all available in one spot here. They're even conveniently all linked to each other so you can go back and forth (although this is not necessarily the chronological order, since point updates may be made to older versions even after a new version has been released).

There are two parts to making your code work with Python 3

In my not terribly extensive experience so far, in the general case porting your code to Python 3 is really two steps in one, not a single process. First, you need to revise your code so that it runs on Python 3 at all; it uses print(), it imports modules under their new names, and so on. Some amount of this can be automated by 2to3 and similar tools, although not all of it. As I discovered, a great deal of this is basically synonymous with modernizing your code to the current best practice for Python 2.7. I believe that almost all of the necessary changes will still work on Python 2.7 without hacks (certainly things like print() will with the right imports from __future__).

After your code will theoretically run at all, you need to revise your code so that it handles strings in Unicode, and it means that calling this process 'porting' is not really a good label. The moment you deal with Unicode you need to consider both character encoding conversion points and what you do on errors. Dealing with Unicode is extra work and confronting it may well require at least a thorough exploration of your code and perhaps a deep rethink of your design. This is not at all like the effort to revise your code to Python 3 idioms.

(And some people will have serious problems, although future Python 3 versions are dealing with some of the problems.)

Code that has already been written to the latest Python 2.7 idioms will need relatively little revision for Python 3's basic requirements, although I think it always needs some just to cope with renamed modules. Code that was already dealing very carefully with Unicode on Python 2.7 will need little or no revision to deal with Python 3's more forced Unicode model, because it's already effectively operating in that model anyways (although possibly imperfectly in ways that were camouflaged by Python 2.7's handling of this issue).

The direct corollary is that both the amount and type of work you need to do to get your code running under Python 3 depends very much on what it does today with strings and Unicode on Python 2. 'Clean' code that already lives in a Unicode world will have one experience; 'sloppy' code will have an entirely different one. This means that the process and experience of making code work on Python 3 is not at all monolithic. Different people with different code bases will have very different experiences, depending on what their code does (and on how much they need to consider corner cases and encoding errors).

(I think that Python 3 basically just works for almost all string handling if your system locale is a UTF-8 one and you never deal with any input that isn't UTF-8 and so never are confronted with decoding errors. Since this describes a great many people's environments and assumptions, simplistic Python 3 code can get very far. If you're in such a simple environment, the second step of Python 3 porting also disappears; your code works on Python 3 the moment it runs, possibly better than it did on Python 2.)