Wandering Thoughts archives

The Spamhaus DBL does get hits even with basic checks

The Spamhaus DBL is unlike their other blocklists in that it is for host and domain names, not IP addresses. As Spamhaus describes it:

The Spamhaus DBL is a realtime database of domains (typically web site domains) found in spam messages. Mail server software capable of scanning email message body contents for URIs can use the DBL to identify, classify or reject spam containing DBL-listed domains.

The intended primary use of the DBL is for message body scanning; you'd identify the hosts mentioned in URLs or URL-like things and run them past the DBL. You can also use it to check hostnames that appear in envelope information, like MAIL FROM (and EHLO, and simply the DNS name), but the way Spamhaus has written it up suggests that this is not going to get very many hits.

(The DBL is not the only such domain based blocklist, of course.)

A while back I added DBL checking to my sinkhole SMTP server and then turned it on, checking all of the MAIL FROM domain, the EHLO name, and the reverse DNS of the connecting IP. I didn't really expect it to get any hits; I basically wanted to experiment. The result contained two surprises.

The first surprise was that even in my modest little context, I see more than a few DBL hits. It's nowhere near the level of the SBL in general (especially the SBL CSS), which I check first, but it does happen enough that it's easy to find rejections that are due to it. This suggests that I should look into using the DBL along side the SBL in our real mail system's spam filtering.

(I want to do some actual analysis there, but that'll be another entry.)

The second surprise is that a lot of the mail senders using DBL listed domains were and are sending from their own servers, and those servers were not listed in the SBL or in fact any of Spamhaus's IP based DNSBLs. Often these people seem to have been sending from the same IP address for quite a while. This is very much not what I expected; I expected that if you were a DBL listed operation, your sending servers would wind up listed in the SBL in short order for, well, sending spam. Instead I see a number of persistent DBL-listed senders with their own static server IPs who are (still) not SBL listed.

(Often the IP addresses aren't even on very many other DNS blocklists, at least out of the ones that I check these days.)

This matters to me because one of the reasons I expected the DBL to have a low (additional) hit rate for things like MAIL FROM checks was that I thought there would be a much bigger overlap between the SBL and the DBL than there is. This expectation of low hit rates is why I haven't really looked even simple DBL usage before now.

(The moral, obviously, is to validate my anti-spam feelings instead of just assuming. A more general moral is that I should think about general infrastructure for doing experiments to measure potential hit rates on things like this. Some amount of things can be looked at in retrospect based on logs, but not everything.)

Some things I believe about importance and web page design

Here are some things that I believe about web page design.

If you have a bright red element on a page that does not otherwise use red as part of its colour scheme, this element should not merely be the most important thing on the page, it should be so important that the reader must look there first and foremost. In at least the west, we're habituated to red as an alert colour, probably the primary alert colour. A solitary red thing is practically screaming at you, jumping up and down and demanding your attention. It should be worth it and really, it had better be.

If you have a bright red element that you lock as a floating element so that it's always present even when the page scrolls, this element should be so important that the reader needs to always pay attention to it. A constantly present red element is a constant alert yelling at you. I actually tend to think that if it is so important, maybe there shouldn't be anything else on the page (or at least no other content).

(In fact, in general any always present element should be very important. Screen space is a precious resource on at least mobile devices (which are an increasing amount of web browsing), so any space you reserve for headers, footers, sidebars, and so on is space that shrinks an already small content area even smaller. People with small screens can get quite irritated about this and yes, they would much rather scroll up to get back to your navigation than have it take up a quarter or a fifth of their screen all the time.)

If you have or are tempted to have a red element on your pages, especially an always visible one, I very strongly think that you should be considering whether it really is that important. Is it an alert that visitors should be paying some amount of attention to all the time, to the degree that some of them may find the result unreadable? Is it more important than the actual content of the web page? Or is it perhaps somewhat less important than that, and so maybe it should not be red, or always visible, or especially not both at once.

(My personal view is that there is very little information that is that important. Your web pages are hopefully about your content, after all, at least for normal websites. About the only case for such an all-consuming alert that I can come up with is content that is actually now dangerous and is being retained only for historical reasons; here, people not reading the content is a feature instead of a bug.)

In not unrelated news, the PowerDNS docs web pages for the 3.x release currently look like this [png]; 3.x is not the latest release, but it is one that is still very much out in the field (it's in Ubuntu 14.04 LTS and Fedora 23, for example). I personally do not consider 'you are reading the 3.x documentation, here is the latest documentation' to be the most important and pretty much all-consuming thing on the page, but it's not my website.

(Nor will I be even attempting to send in a patch, because the only patch I would write is one that deleted the entire 'this is older documentation' alert. There is no possible way of fixing this within anything that looks and acts like the current setup, and I have to assume that the people who created the alert feel that it is really that important. In that case we have a disagreement not about presentation, which can be patched with HTML and CSS, but about web page information architecture.)