Wandering Thoughts archives

Why I am not really interested in hearing blacklist appeals

From a comment on a previous entry:

There is considerable merit in allowing blacklisted sites to contact you to let you know that you've blacklisted them in error.

I'm not really convinced of this; especially I am not really convinced that it's at all useful to make it easy to do so from the outside, for example by never blocking email to postmaster. For a start, if a blocked site contacts me to say 'we are trying to mail your users but you are rejecting us', how am I supposed to know that they are not lying?

Really, the only people I want to listen to about this are my users, so I want my users themselves to tell me 'some email that I want is not getting through'. If an outside site wants to get un-blocked, they're best off by getting in touch with whichever of our users they're trying to mail and having that user ask us to fix the situation.

(Pragmatically, anyone who really wants to get through an email blocklist has lots of ways that don't even cost money, for example sending from Google Mail, so it should not be hard for such places to reach our users to let them know.)

Even if I was mandated to allow blacklisted sites to directly contact us, I do not think I would do it by email, and especially not by a well known common email address like postmaster, because well known email addresses invariably get hit by spam, so I would expect almost all email to postmaster to be spam, which is not a good recipe for spotting the one appeal email in five hundred spams. I think that a far better way is to use a web form or some other non-email method; if you have to use an email address, it should be specific to your site and probably change every so often. (Put the URL, or the email address, in the text of the SMTP error message.)

Why mail systems should not defer rejections to RCPT TO time

There is a movement for the default configurations of things like exim to defer sender verification to RCPT TO time; instead of reporting an error or a defer after the MAIL FROM, all MAIL FROMs are accepted and only later does the message start getting errors. I have recently come to a realization about why this is wrong, and I even have an example.

The problem is that when you give at least a 4xx error to an RCPT TO, it makes the sending mailer think that there is a problem with that RCPT TO address, not with the MAIL FROM address. The sending mailer may then sensibly defer all email to that recipient, because after all you told it that there was a problem with that address. (The actual text of your 4xx error may explain the situation, but mailers don't yet read English error messages.)

We have actually seen this happen with email from our central campus mail system for someone who was forwarding their email to our system. Some spam domain fell out of the DNS between the central mail system accepting it and it coming to us, we started giving temporary defers at RCPT TO time, and all mail for this person backed up.

I believe that this is done because people feel that some mailers do not react well to MAIL FROM errors (and I've occasionally seen evidence of that in our logs). However I feel that the cure is worse than the disease, and such bad mailers are clearly violating the specification to start with; coddling spec-violating mailers while causing problems for mailers that are following the spec does not seem like a good tradeoff for me.

(Besides, we ran our system with sender verification problems reported at MAIL FROM time for years without getting any complaints or problem reports, so we have empirical evidence that it works fine.)

Theoretically this also allows you to accept mail for postmaster whether or not the sender address actually exists. Personally I do not believe that this is actually a feature, especially since it has been years since we got any legitimate outside email to postmaster; what we do get has been spam.

Weekly spam summary on October 13th, 2007

This week, we:

• got 11,905 messages from 252 different IP addresses.
• handled 27,710 sessions from 2,367 different IP addresses.
• received 342,122 connections from at least 124,401 different IP addresses.
• hit a highwater of 36 connections being checked at once.

Connection volume seems up a bit from last week, although it's hard to be entirely sure. Session volume is definitely up, pretty much to the level it was two weeks ago.

Day	Connections	different IPs
Sunday	52,106	+22,241
Monday	72,645	+27,772
Tuesday	47,247	+16,403
Wednesday	33,365	+13,620
Thursday	52,521	+21,076
Friday	48,166	+12,650
Saturday	36,072	+10,639

It's interesting that this seems to vary all over the map from day to day, and it amuses me that Wednesday, for long the most active day, is the least active day this week.

Kernel level packet filtering top ten:

Host/Mask           Packets   Bytes
213.180.130.0/24      22255   1335K onet.pl
72.249.13.64/26       14924    819K otcpicknews.com
68.230.240.0/23       12994    631K cox.net
213.4.149.241         10710    571K
218.0.0.0/11           8620    419K CHINANET
68.99.120.0/24         8496    400K coxmail.net
204.127.225.0/24       6321    405K comcast.net
206.18.177.0/24        6146    393K comcast.net
213.29.7.0/24          5579    335K centrum.cz
209.51.135.180         5141    282K

Volume is down a bit from last week, but not really significantly, and once again almost of the top 10 is netblocks.

• 213.4.149.241 kept trying with bad HELOs; we saw it before in August.
• 209.51.135.180 kept trying to send us mail with an origin address that had already tripped our spamtraps.

Connection time rejection stats:

 111794 total
  54499 bad or no reverse DNS
  47536 dynamic IP
   5567 class bl-cbl
    973 class bl-pbl
    458 class bl-dsbl
    317 qsnews.net
    296 class bl-sbl
    280 class bl-sdul
    149 class bl-njabl
    129 dartmail.net
    125 acceleratebiz.com

The highst source of SBL rejections this week is SBL56712 with 94 rejections (a /28 listed as a spam source for power-cl1cks.com, listed in July), followed by SBL59518 with 79 rejections (a /24 also for 'power-cl1cks2.com'), and SBL58952 with 33 rejections (a /27 from September, 'spwu10.net'). I've seen other spwu10.net machines crop up from 74.223.112.0/22, so I think it and them are going into our overall blocklists.

(A modest suggestion to people: do not give your domains sequence numbers. It does not really look good.)

Eight of the top 30 most rejected IP addresses were rejected 100 times or more this week; the leader is 200.186.145.197 (1,259 rejections), followed by 200.177.119.109 (388 rejections). Oddly enough, none of the top 30 appear to be showing up on any of the popular DNS blocklists this week; this seems implausible, which means that something is broken somewhere.

(Locally, 16 were rejected for being dynamic IP addresses, 11 for having bad or missing reverse DNS, 2 for being qsnews.net, and 1 for being qsc.de.)

This week, Hotmail had:

• no messages accepted.
• no messages rejected because they came from non-Hotmail email addresses.
• 49 messages sent to our spamtraps.
• 2 messages refused because their sender addresses had already hit our spamtraps.
• 4 messages refused due to their origin IP address (one in the CBL, one from Nigeria, one from Ghana, and one from saix.net aka telkom.co.za).

And the final numbers:

what	# this week	(distinct IPs)	# last week	(distinct IPs)
Bad HELOs	6739	363	1751	270
Bad bounces	669	553	114	78

The leading source of bad HELOs this week was 208.223.173.169 (243 attempts), followed by 202.155.205.242 (123 attempts), and 216.157.197.66 (91 attempts). There are a lot of people with relatively high counts (above 50 attempts), which is not really surprising given the stats.

Bad bounces were sent to 650 different bad usernames this week, with the most popular one being Jayce_Pirani with 5 attempts, followed by HoratioClemens with 4 attempts and MaxwellFocke and last week's winner SHOUGEE with 3 attempts each. There was one attempt to the all-number bad username 405 and one to "Gresham," (sic), and some to ex-users, but with 650 of them I'm not going to study them carefully enough to draw real conclusions.

Weekly spam summary on October 6th, 2007

Unfortunately, our SMTP frontend died Thursday afternoon, so some of our usual stats are approximations or partial stats. Having said that, this week we:

• got 11,577 messages from 283 different IP addresses.
• handled 20,711 sessions from 1,929 different IP addresses.
• received at least 317,396 connections from at least 73,000 different IP addresses.
• hit a highwater of 38 connections being checked at once.

In specific, we got 184,251 connections from at least 73,605 different IP addresses through Thursday morning at 4am, and then 133,145 connections from at least 52,709 different IP addresses since 2:40pm Thursday. Connection volume is up a bit from last week.

Kernel level packet filtering top ten:

Host/Mask           Packets   Bytes
72.249.13.64/26       36754   2016K otcpicknews.com
213.180.130.0/24      27166   1630K onet.pl
213.29.7.0/24          9983    599K centrum.cz
68.230.240.0/23        7805    379K cox.net
68.168.78.0/24         6797    326K adelphia.net
71.165.18.155          6369    298K
204.127.225.0/24       6077    389K comcast.net
206.18.177.0/24        5737    367K comcast.net
70.60.187.42           5008    240K
218.0.0.0/16           4897    235K CHINANET

Total volume is slightly up from last week. Strikingly, only two of the top ten this week are individual IP addresses, although this is the first time in a while that a large netblock has made the top ten.

• 71.165.18.155 kept trying to send us phish spam that had already tripped our spamtraps.
• 70.60.187.42 is on the DSBL.

Connection time rejection stats:

 114152 total
  52897 dynamic IP
  52569 bad or no reverse DNS
   5520 class bl-cbl
   1119 class bl-pbl
    309 class bl-sdul
    309 class bl-dsbl
    161 acceleratebiz.com
     87 qsnews.net
     85 class bl-sbl
     75 class bl-njabl
     53 officepubs.com

Volume is up significantly from last week. The highest source of SBL rejections this week was the same as last week; SBL58952, with 22 rejections, followed by SBL39831 with 20 rejections (spam emitters since 23 May 2006) and SBL48694 with 10 rejections (also returning from last week).

Nine of the top 30 most rejected IP addresses were rejected 100 times or more this week; the leader is 88.245.33.111 (527 rejections), followed by 59.93.10.75 (241 rejections) and 85.101.255.175 (230 rejections). Fifteen of the top 30 are currently in the CBL, two are currently in bl.spamcop.net, sixteen are in the PBL, and a grand total of 18 are in zen.spamhaus.org.

(Locally, 23 were rejected for bad or missing reverse DNS, 4 for being something we considered a dynamic IP address, 1 for being qsnews.net, 1 for being in AccelerateBiz space, and one for being in the DSBL.)

This week, Hotmail had:

• 1 message accepted.
• no messages rejected because they came from non-Hotmail email addresses.
• 41 messages sent to our spamtraps.
• 2 messages refused because their sender addresses had already hit our spamtraps.
• 8 messages refused due to their origin IP address (four from the Cote d'Ivoire, two from Ghana, one from saix.net, and one in the CBL).

And the final numbers:

what	# this week	(distinct IPs)	# last week	(distinct IPs)
Bad HELOs	1751	270	5489	399
Bad bounces	114	78	1521	1115

There is no particularly big source of bad HELOs this week; the top single source only made 36 attempts.

Bad bounces were sent to 83 different bad usernames this week, with the most popular one being Harjas_Muthukumar with 15 attempts, followed by ToddWolseley with 7 attempts and the now-familiar SHOUGEE with 4 attempts. Other representative bad usernames include natukida, tuncer784, zddzqdekcztiu, and mari-tachi, along with a number of ex-users; the leading form seems to be the FirstLast one.

The leading single source of bad bounces this week is actually a German site, but ezweb.ne.jp and softbank.ne.jp are up near the top plugging away. Google seems to have given us a miss this week, although various .edu sites that should really know better made up for them. My pick for the most amusingly named source this week is xmldove.fastfreenet.com, a name that puts all sorts of amusing and peculiar images into my head.