Wandering Thoughts archives

Code stability in my one Django web application

We have one Django web application, a system for automating the handling of much of our new Unix account requests. It was started in early 2011 (using Django 1.2) and I did a retrospective at the end of 2014 where I called it a faithful web app, one that had just kept on quietly working without problems. That's continued through to today; the app needs no routine attention, although every so often I tweak it to better handle an obscure situation.

One of the interesting aspects of that quiet stability is the relative stability of the application's Python code over those nearly six years so far. There are web frameworks where in six years you'd need to significantly rework and restructure your code to deal with changing APIs and approaches. For us, Django hasn't been one of them. Although we're not quite current on Django versions, we're not that far back, yet much of the code is basically the same (or literally the same) as it started out all those years ago. I'm pretty sure that almost all of our model and view code is untouched over that time, and I think a lot of our templates are untouched or only minorly changed.

However, this is not a complete picture of code churn in our app, because there have been Django changes over that time in areas such as routing, command argument processing, template processing, and project structure. These changes have forced code changes in the areas of our app that deal with such things (and the change in project structure eventually forced a massive renaming of files when we went to Django 1.9). While this sounds kind of bad, I've wound up considering all of them to be relatively peripheral. In a way, all of the code involved is plumbing and glue. None of it really touches the heart of our web application, which (for us) lives mostly in the models and views and somewhat in the core logic of the templates. Django has been very good about keeping that core code from needing any substantive changes. We still validate form submissions and generate views and process model data in basically the same way we did in 2011, and all of that is what I think of as the hard stuff.

(Although I haven't measured, I think also it's most of the app's code by line count.)

This code stability is one reason why Django upgrades have been somewhat painful but not deeply painful. If we'd needed major code restructuring, well, I'd probably have done it eventually because we might have had no choice, but we'd have likely updated Django versions more sporadically than we have so far.

PS: Although Django is going from version 1.11 to version 2.0 in the next release, the Django people say that this shouldn't be any more of an upgrade than usual. And speaking of that. I should get working on updating us to 1.11, since security updates for 1.10 will end soon (if they haven't already).

The dig program now needs some additional options for useful DNS server testing

I've been using the venerable dig program for a long time as my primary tool to diagnose odd name server behavior. Recently, I've discovered that I need to start using some additional options in order for it to make useful tests, where by 'useful tests' I mean that dig's results correspond to results I would get through a real DNS server such as Unbound.

(Generally my first test with DNS issues is just to query my local Unbound server, but if I want to figure out why that failed I need some tool that will let me find out specific details about what didn't work.)

For some time now I've known that some nameservers reject your queries if you ask for recursive lookups, so I try to use +norecurs. In exploring an issue today, I discovered that some nameservers also don't respond if you ask for some EDNS options, which it turns out that dig apparently now sets by default. Specifically they don't respond to DNS queries that include an EDNS COOKIE option, although they will respond to queries that are merely EDNS ones without the COOKIE option or any other options.

(Some experimentation with dig suggests that including any EDNS option causes these DNS servers to not respond. I tried both +nocookie +nsid and +nocookie +expire, and neither got a response.)

This means that for testing I now want to use 'dig +norecurs +nocookie', at least. It's possible that I want to go all the way to 'dig +norecurs +noedns', although that may be sufficiently different from what modern DNS servers send that I'll get failures when a real DNS server would succeed. I expect that I'm going to want to wrap all of this in a script, because otherwise I'll never remember to set all of the switches all of the time and I'll sometimes get mysterious failures.

(Some experimentation suggests that my Unbound setup sends EDNS0 queries with the 'DNSSEC Okay' bit set and no EDNS options, which would be 'dig +norecurs +nocookie +dnssec' if I'm understanding the dig manpage correctly. These options appear to produce DNS queries that the balky DNS server will respond to. With three options, I definitely want to wrap this in a script.)

What this suggests to me in general is that dig is not going to be the best tool for this sort of thing in the future. The Dig people clearly feel free to change its default behavior, and in ways that don't necessarily match what DNS servers do; future versions may include more such changes, causing more silent failures or behavior differences until I notice and carefully read the manpage to find what to turn off in this new version.

(A casual search turns up drill, which is another thing from NLNet Labs, the authors of Unbound and NSD. Like dig, it defaults to 'recursion allowed' queries, but that's probably going to be a common behavior. Drill does have an interesting -T option to do a full trace from the root nameservers on down, bypassing whatever your local DNS resolver may have cached. Unfortunately it doesn't have an option to report the IP address of the DNS server it gets each set of answers from; you have to go all the way to dumping the full queries with -V 5.)