Today's lesson on the value of commenting your configuration settings
We have a relatively long-standing Django web application that was first written for Django 1.2 and hadn't been substantially revised since. Earlier this year I did a major rework in order to fully update it for Django 1.9; not just to be compatible, but to be (re)structured into the way that Django now wants apps to be set up.
Part of Django's app structure is a
settings.py file that contains,
well, all sorts of configuration settings for your system; you
normally get your initial version of this by having Django create
it for you. What Django wants you to have in this file and how it's
been structured has varied over Django versions, so if you have a
five year old app its
settings.py file can look nothing like what
Django would now create. Since I was doing a drastic restructuring
anyways, I decided to deal with this issue the simple way. I'd have
Django write out a new stock
settings.py file for me, as if I was
starting a project from scratch, and then I would recreate all of
the settings changes we needed. In the process I would drop any
settings that were now quietly obsolete and unnecessary.
(Since the settings file is just ordinary Python, it's easy to wind up setting 'configuration options' that no longer exist. Nothing complains that you have some extra variables defined, and in fact you're perfectly free to define your own settings that are used only by your app, so Django can't even tell.)
In the process of this, I managed to drop (well, omit copying) the
ADMINS setting that makes Django send us email if there's an
uncaught exception (see Django's error
I didn't spot this when we deployed the new updated version of the
application (I'm not sure I even remembered this feature). I only
discovered the omission when Derek's question here
sent me looking at our configuration file to find out just what
we'd set and, well, I discovered that our current version didn't
have anything. Oops, as they say.
Looking back at our old
settings.py, I'm pretty certain that I omitted
ADMINS simply because it didn't have any comments around it to tell
me what it did or why it was there. Without a comment, it looked like
something that old versions of Django set up but new versions didn't
need (and so didn't put into their stock
settings.py). Clearly if I'd
checked what if anything
ADMINS meant in Django 1.9 I'd have spotted
my error, but, well, people take shortcuts, myself included.
(Django does have global documentation for settings, but there is no global alphabetical index of settings so you can easily see what is and isn't a Django setting. Nor are settings grouped into namespaces to make it clear what they theoretically affect.)
This is yet another instance of some context being obvious to me
at the time I did something but it being very inobvious to me much
later. I'm sure that when I put the
ADMINS setting into the initial
settings.py I knew exactly what I was doing and why, and it was
so obvious I didn't think it needed anything. Well, it's five years
later and all of that detail fell out of my brain and here I am,
re-learning this lesson yet again.
(When I put the
ADMINS setting and related bits and pieces back
settings.py, you can bet that I added a hopefully clear
comment too. Technically it's not in
settings.py, but that's
a topic for another blog entry.)
If you send email, don't expect people to help you with abuse handling
I'll start with the tweets:
@thatcks: I see these spammers used @MailChannels to hit us once before, in April. I reported them then, but I have no time for this shit any more.
Back in April, a persistent long-term spammer of one of our addresses attempted to send it spam via MailChannels, a commercial email sending outfit. I complained to MC's abuse contacts at the time, because I'm an optimist, and someone at MC got back to me to tell me this spammer had been fixed. Then they came back now (well, a couple of days ago).
@thatcks: As has been said many, many times before, expecting the receivers of email to be your anti-spam detection method is utterly broken.
Some people might say that I should do the 'responsible' thing and once again report this incident to MailChannels. These people are wrong. It is always the sender's responsibility to detect that they are sending spam and take steps to deal with it; as has been said many years ago, abuse reports are a gift (one that comes from fewer and fewer people these days). In my case, my only real interest is in making the spam stop and generally I have far more effective ways of doing this than sending in complaints.
(By the way, I hope we can agree that there is absolutely no moral basis for saying that people have a responsibility to report spam. If your service is spamming me, I am getting absolutely nothing out of this and I accordingly owe you absolutely nothing. In fact, morally speaking you owe me for inflicting costs on me.)
In this specific situation, it's also clear that sending in complaints is not effective (cf). After all, I already did that once, got an assurance that it was dealt with, and the spammer came back a couple of months later. A repeat report is likely to net exactly the same result at best.
Then MailChannels popped up:
@MailChannels: @thatcks We don't take abuse of our network lightly and are keen to investigate. Please send us sample messages to email@example.com
This is a form tweet. It betrays at least an inability to read my original message.
(Replying to aggravated people with form tweets that betray a lack of thinking human involvement is, at the least, going to aggravate them further. So it proved here.)
@thatcks: .@MailChannels You're asking me to do more work to help you out. Why would I do that? If you want, you have enough information already.
I gave the form tweet all the response I felt that it deserved. And it's true that MailChannels has all the information they need; they could just search their April abuse reports for my name, find the address here that I reported was hit, and see if that address was sent to recently. Why yes, yes it was. MailChannels' email to it was even rejected this time around too, which really ought to be one of a number of danger signs for MailChannels. Certainly this would take some work on MailChannels' part, but you know, they're the people that this benefits, not me; I've already taken effective steps on our side.
(MailChannels benefits because they get rid of a spammer who may drag their reputation down and damage the deliverability of email for other paying customers, which would cost MailChannels money.)
Of course, I expect that MailChannels did nothing here. That's the easy way to blow off problem indicators while feeling good about yourself; you can say 'well, if it was real the person would have totally taken us up on our offer'. They can tick off the 'we tried' box and consider the matter done. And really, what mail sending service can afford to actually do a good job with spam?
(Applications of this pattern to, say, bug reports and bug trackers are left as an exercise for the reader.)