Wandering Thoughts archives

A Django application design puzzle about modularity

As I mentioned in passing yesterday, we need a web-based email settings control panel for our users, something that allows them to set various options for their account in more user-friendly ways than editing their .forward files or logging in to our Unix machines to run scripts. The natural way for me to put this together is to use Django, since I've been happy with our other Django-based web application.

But this now leaves me with a design puzzle. We'd really like this email settings control panel to be 'modular' in the sense that we can easily add more settings to it over time, exposed through some UI and the like (in fact it's pretty much a requirement that it be modular this way). However it's not clear to me how to structure this in Django, either in the project structure or in the database. The most obvious setup for code modularity probably wants each separate module of options to be a separate web page on a distinct URL. This is simple to implement but kind of a terrible user experience (and it results in a proliferation of database tables).

A more elaborate version would embed all of the forms on the same web page, probably manipulating various sorts of form visibility and form submission through JavaScript for a better user experience. But that feels like it's going to require a bunch of hacking, likely embeds knowledge of all of the modules (and all of the active modules for a given user) into the page's template, and so on.

Finally there's a less Django-native version of modularity where the overall form presented to users is not a representation of a model. Instead we'd some database of settings (probably very abstracted) and the form code would basically do manual mapping between the database and what the user saw. We'd have our own plugin system so that a setting 'module' could define its settings, how to represent them in a form, and so on. However theoretically right this sounds the drawback here is that is that I'd be writing a lot of code and designing completely custom systems.

(I could even throw out Django in favour of a simpler web framework system, which makes it clear how far out this approach is.)

I don't have any answers to this puzzle right now because I've only just started really thinking about it. It's quite possible that Django has some clever way of doing or supporting this that I haven't run into yet, or that there's a common design pattern in the Django community for handling this issue.

(I certainly hope that there turns out to be an obvious and simple way to do this inside Django; it'd make my life simpler. While one part of me wouldn't mind diving head-first into building my own modularity framework, another part of me wants to get this project done in a sane amount of time.)

Python 3 has very little benefit for ordinary Python programmers

Sometimes an incompatible transition is strongly justified. In some cases the old code and the old ways were actively dangerous to people because they were too easy to misuse (or were actually basically impossible to use safely); in other cases the baggage of the old was making it essentially impossible to add important new features that people actively wanted.

The Python 3 transition is not one of these. It was almost entirely about removing warts in the language, and here is the thing: ordinary programmers don't really care about language warts. Every language has some warts and in practice those warts rarely get in the way of doing work in the language; people work around them if necessary and often don't even notice them. Removing these warts from Python was (as far as I can tell) not required to make other progress in the language or the standard library. They were just things about the language that irritated the core Python developers.

(Hence, among other things, the comparison of Python 3 to XHTML.)

The big exception to this is also the most prominent and consequential change in Python 3, that of making strings into Unicode by default. But as Python 2's 'from __future__ import unicde_literals' demonstrates, this did not have to be an incompatible change; it could have been put in place in stages.

(In fact anything that is now covered in 'from __future__ import ...' could have been implemented in stages, just as many past Python transitions have been managed.)

Note that this is not the same thing as saying that Python 3 has not brought new and worthwhile things to Python programmers. It certainly has. But as far as I can tell the reason they are only in Python 3 is a choice on the part of the Python developers, not a requirement.

(This idea is not unique to me by any means and I've touched on it in passing before, but today I want to state it explicitly.)

The core issue in the Python 3 transition (from my view)

In response to my entry about how Python 3 has always made me kind of angry, a commentator asked an interesting question:

You're right. Python developers aren't the first people to deprecate a heavily-used product. Is there anyone who has made an incompatible transition like this whom you can point to as a good example to follow?

[...]

So, what do you do as a developer? What's a better way to shed the cruft of some bad choices while not making people angry and not having to keep adding new features to the code that has given you headaches for so long?

As I read it, this question contains a hidden assumption: that you are going to make an abrupt and thus incompatible transition. I don't think that there's any good way to do this in a language and I don't know of any languages that have managed it gracefully once they got a significant number of users. An incompatible transition by definition creates not one language but two closely related languages, possibly somewhat translateable.

(It's theoretically possible to successfully do a transition like this; what you need is a tool that mechanically rewrites old code to be new, working code. Go actually has such a thing for many language and library changes. 2to3 was not such a tool.)

Such transitions are almost always the result of choices, or really one choice, that of the developers choosing to walk away from the old code. If you refuse to do any work to have a graceful transition then of course you don't get one. This is more or less what I remember the Python developers doing, at least initially; Python 2.7 people had a few limited bones thrown to them but not really all that many. In theory I think it would have been perfectly possible to do a much more graceful transition between Python 2 and Python 3. It just would have taken more time and required adding more things to Python 2 (and probably to its standard library).

(For 'code' one should really read 'language design'. I don't think that the actual CPython code base underwent any particularly major upheavals and rewrites between Python 2.7 and Python 3 and all of the issues that the Python developers say prompted Python 3 were about historical warts in the language.)

There's more I could say on this but in the end none of it matters. The Python developers made a decision that they were not interested in doing further work on Python 2.7 and users of Python could more or less lump it. If the developers are not interested in a graceful transition, you do not get one.