Wandering Thoughts archives

Weekly spam summary on December 30th, 2006

The SMTP frontend died twice late Friday night, to my irritation. That said, this week we:

• got 9,806 messages from 186 different IP addresses.
• handled 15,551 sessions from 916 different IP addresses.
• received something over 204,995 connections from at least 58,611 different IP addresses.
• hit a highwater of at least 6 connections being checked at once.

The message count is down dramatically from last week because this week is a vacation week for the university. The connection volume is not down at all because spammers don't really take vacations.

Kernel level packet filtering top ten:

Host/Mask           Packets   Bytes
213.4.149.12          10174    529K
71.16.74.210           7322    337K
193.252.22.158         6772    406K
213.29.7.0/24          5738    344K
210.92.140.22          4222    203K
217.12.180.25          3418    164K
196.25.78.162          2679    135K
213.4.149.66           2279    119K
212.43.241.13          2247    123K
203.97.33.64           2205    115K

This is somewhat quieter than last week, and centrum.cz's /24 continues to drop in the league tables.

• 213.4.149.12, 193.252.22.158, and 212.43.241.13 return from last week.
• 71.16.74.210 and 203.97.33.64 kept trying to send us stuff that had already tripped our spamtraps.
• 210.92.140.22 and 196.25.78.162 had bad DNS.
• 217.12.180.25 and 213.4.149.66 kept trying bad HELOs.

(The symmetry here is amusing but random.)

Connection time rejection stats:

  64823 total
  36173 dynamic IP
  22508 bad or no reverse DNS
   4600 class bl-cbl
    349 class bl-dsbl
    279 class bl-sdul
    202 'fairgamemail.us'
     83 class bl-njabl
     48 class bl-sbl
     38 class bl-spews
     36 cuttingedgemedia.com

Good old Cutting Edge Media. Too cutting edge to catch a clue, evidently. (Although it's more likely that they're merely uninterested in purging bouncing addresses from their mailing lists, because that would reduce their nominal value.)

What I have marked down in our records as 'fairgamemail.us' is 204.14.1.66 to 204.14.1.126, which is all part of SBL27197 (a /21 for '247 Surf Net' or cpchosting, listed since October 29th) and which, judging from the DNS names, may belong to something with the inviting name of 'optindirectmail'. (They've hit us in previous weeks; I don't always bother sorting through the explicitly blocked people to boil out a nice report.)

Only two out of the top 30 most rejected IP addresses were rejected 100 times or more this week; 124.240.124.166 (139 times) and 196.25.78.162 (121 times). 17 of the top 30 most rejected IP addresses are currently in the CBL, 5 are currently in bl.spamcop.net, and one of them is in the SBL; 209.205.236.225, which is part of SBL41018 and SBL49194 (the former is a /24 for a spammer, the latter is a /20 for pacnet.com.mx spammer hosting as an escalation listing, both dating from December 24th).

Pretty much as usual, 209.205.236.225 was not actually rejected for being on the SBL but for other stuff that we check first, in this case missing reverse DNS. Possibly I ought to promote the SBL and so on to somewhat earlier in the checking. The top actual SBL rejections were:

11	SBL38413	a /20 for ServerFlo, Inc (23 Nov 2006, although all our hits were in a /24 that is SBL37655, which was listed 12 Oct 2006)
10	SBL49248	Vietnamese webmail that is an advance fee fraud spam source (18 Dec 2006)
5	SBL49074	hijacked server sending spam (13 Dec 2006)

(Plus something that has since been removed from the SBL, so I'm not going to mention it.) |

This week, Hotmail managed:

• 3 messages accepted, which I really suspect were all spam.
• no messages rejected because they came from non-Hotmail email addresses.
• 12 messages sent to our spamtraps.
• no messages refused because their sender addresses had already hit our spamtraps.
• 1 message refused due to its origin IP address being in the CBL.

Okay, maybe Hotmail's advance fee fraud spammers take small vacations.

And the final numbers:

what	# this week	(distinct IPs)	# last week	(distinct IPs)
Bad HELOs	594	87	1147	104
Bad bounces	72	60	117	98

There are no particularly outstanding sources of bad HELOs this week, nor any particularly clear locus of bad bounces the way there was last week. Random alphabetical usernames made up most of the bounce targets, with the leader being 'ijiefuurmcl' (five bounce attempts).

Weekly spam summary on December 23rd, 2006

The SMTP frontend died and was restarted at 2:24 am Wednesday morning, so some stats are only from then. That said, this week we:

• got 14,896 messages from 260 different IP addresses.
• handled 22,673 sessions from 1,353 different IP addresses.
• received 147,470 connections from at least 47,766 different IP addresses since Wednesday at 2:24 am.
• hit a highwater of 10 connections being checked at once since Wednesday at 2:24 am.

It looks like we'd received about 65,000 connections as of Tuesday morning, which would make the total volume roughly the same as last week. The per-day information is kind of broken, but since Wednesday morning we seem to have had higher traffic than usual, running between 35,000 and 40,000 connections a day.

Kernel level packet filtering top ten:

Host/Mask           Packets   Bytes
62.94.0.34            13437    605K
213.4.149.12          12301    640K
213.29.7.0/24          8888    533K
193.252.22.158         4936    296K
217.20.114.13          4783    287K
216.48.45.2            3515    164K
212.43.241.13          3478    191K
60.231.152.85          3241    165K
83.17.193.138          3045    183K
62.66.138.173          2939    149K

This is a change from last week, with totallyfreeld.net dropping out completely and a welcome drop in overall volume.

• 62.94.0.34 and 216.48.45.2 had too many bad HELOs.
• 213.4.149.12, 193.252.22.158, and 60.231.152.85 return from last week.
• 217.20.114.13 is in the NJABL.
• 212.43.241.13 is a fr.clara.net machine that kept on trying to send us stuff that had already tripped spamtraps.
• 83.17.193.138 and 62.66.138.173 are dynamic IP 'dialup' machines.

Connection time rejection stats:

  52591 total
  31494 dynamic IP
  16375 bad or no reverse DNS
   3541 class bl-cbl
    312 class bl-sdul
    222 class bl-dsbl
     77 class bl-njabl
     44 class bl-sbl
     24 class bl-spews
     15 cuttingedgemedia.com
      8 class bl-ordb

This is, alas, the last week that the ORDB will appear in the stats, as the ORDB shut down December 18th (as reported on Slashdot, among other places; I am not linking to their website, because they're going to turn that off soon).

Only one out of the top 30 most rejected IP addresses was rejected 100 times or more this week: 63.138.101.141 (102 times, in the CBL). 16 of the top 30 are currently in the CBL and 7 are currently in bl.spamcop.net.

Almost half of the SBL rejections this week came from one IP, 202.175.95.171 aka SBL49074, apparently a hijacked spam sending machine. The next two, rejected five times each, are 66.158.163.165 (SBL49046) and 221.133.1.17 (SBL49248). In a sign that the universe is returning to the proper order of things, both are listed for being advance fee fraud spam sources.

This week, Hotmail had:

• 1 message accepted.
• no messages rejected because they came from non-Hotmail email addresses.
• 28 messages sent to our spamtraps.
• 3 messages refused because their sender addresses had already hit our spamtraps.
• 3 messages refused due to their origin IP address (2 for being in the CBL, one for being in SBL20211 and SBL46450).

And the final numbers:

what	# this week	(distinct IPs)	# last week	(distinct IPs)
Bad HELOs	1147	104	1017	109
Bad bounces	117	98	80	64

By far the champion source of bad HELOs is 195.97.221.30 (335 rejections), followed by 12.162.97.71 (98 rejections). The leading general area for bad bounces seems to have switched to Italian ISPs this week. The random alphabetical names are the leading bad bounce targets, but no one of them particularly stands out.

Something to avoid in callback email address verification

Here's something I would like to grind into various people programming callback email address verification:

Don't do callback email address verification with a MAIL FROM of <> unless the address has actually sent you email.

Why? Because if an address never sends any email to start with, it may not be willing to receive bounces (the major source of email from <>). Blocking null MAIL FROMs from sending email to such addresses is a completely rational way to block bad bounces from forged spam runs.

The people doing this that make me really grind my teeth are SourceForge, who insist that both the origin address of the email and postmaster be willing to accept mail from <>. Our postmaster address is often forged on spam and never sends email, so we would really like to refuse bounces to it. Unfortunately mailing to SourceForge hosted mailing lists is somewhat more important, although we have been known to keep postmaster blocked most of the time and manually unblock it when necessary.

Not that callback email address verification is a good idea in general. But if people are going to implement a non-good idea, I'd like them to do it in a way that doesn't make me grind my teeth in their direction.

(Although every now and then I am tempted to hack something into our mailer configuration to auto-accept every address verification attempt from certain annoying places, like Earthlink and Verizon, no matter whether or not the address actually exists. (If they actually send email, we can refuse it at the DATA phase or something.))

Weekly spam summary on December 16th, 2006

This week, we:

• got 16,689 messages from 271 different IP addresses.
• handled 21,893 sessions from 1,229 different IP addresses.
• received 207,766 connections from at least 62,254 different IP addresses.
• hit a highwater of 8 connections being checked at once.

This is about the same volume as last week, although the number of different IP addresses connecting to us is unusually large.

Day	Connections	different IPs
Sunday	36,194	+11,759
Monday	31,707	+9,762
Tuesday	39,486	+11,117
Wednesday	30,112	+9,187
Thursday	25,760	+7,261
Friday	26,366	+6,996
Saturday	18,141	+6,172

Kernel level packet filtering top ten:

Host/Mask           Packets   Bytes
208.99.198.64/27      29607   1776K
213.29.7.0/24         16473    988K
213.4.149.12           8042    418K
60.231.152.85          6112    311K
69.178.167.2           5894    283K
66.199.252.234         4744    285K
193.252.22.158         4657    279K
195.225.106.170        3907    234K
72.164.45.65           2696    129K
63.138.101.136         2566    123K

• 208.99.198.64/27 is totallyfreeld.net, aka SBL48200, still not terminated by their upstream and still active, returning from two weeks ago.
• 213.29.7.0/24 is the centrum.cz mail servers, returning from last week and still justifying their permanent block.
• 213.4.149.12 and 193.252.22.158 return from last week.
• 60.231.152.85 is a bigpond.net.au cablemodem, and returns from October.
• 69.178.167.2 and 72.164.45.65 tried to send a lot of bad HELOs.
• 66.199.252.234 and 195.225.106.170 tried to keep sending us stuff from origin addresses that had already tripped our spamtraps.
• 63.138.101.136 is in the CBL.

Overall, this week is quieter than last week.

Connection time rejection stats:

  48974 total
  30101 dynamic IP
  13820 bad or no reverse DNS
   3483 class bl-cbl
    271 class bl-sdul
    195 class bl-dsbl
    147 class bl-njabl
     82 class bl-spews
     74 cuttingedgemedia.com
     30 class bl-sbl
     23 class bl-ordb

There was only one IP address out of the top 30 most rejected IP addresses that was rejected 100 times or more, but that was our old friend 64.166.14.222 (631 times). Twelve of the top 30 are current in the CBL, eight are currently in bl.spamcop.net, and one is in the SBL: 213.154.88.54, apparently an ADSL line in Dakar, is in SBL21134 and SBL43951. You win no prizes for guessing that both listings are for being an advance fee fraud spam source.

(Ironically it accounted for none of the SBL hits this week, because it was blocked for having no reverse DNS, and that's checked before the SBL. The lead SBL hit source is 72.5.205.109 at 13 times, in SBL45324 as part of a ROKSO listing for 'Brian Kramer' aka 'Expedite Media Group'.)

This week, Hotmail had:

• 3 messages accepted; I suspect that at least two of them were spam.
• no messages rejected because they came from non-Hotmail email addresses.
• 25 messages sent to our spamtraps.
• 4 messages refused because their sender addresses had already hit our spamtraps.
• 3 messages refused due to their origin IP address being in the CBL.

And the final numbers:

what	# this week	(distinct IPs)	# last week	(distinct IPs)
Bad HELOs	1017	109	785	146
Bad bounces	80	64	109	95

This week there are no bad bounces to 'first_lastname' login names. They've been entirely supplanted by a random parade of alphabetical jumbles, the most popular of which was 'sxdijkhocqn' (5 times), leaved with a few things that are somewhat more plausible usernames, and a couple of old login names that no longer exist.

An unsurprising discovery about spammer behavior

Here's a recent, not entirely surprising discovery about spammer behavior: some spammers are really slow to pick up DNS updates.

We changed MX entries to point to our new SMTP frontend on late Monday afternoon. Our MX entries had the standard 24 hour timeout and our secondary servers had updated to the new zones by Tuesday morning at the latest, so by now it is more than two days after our old MX entries were required to have been purged from caches, even if they were gotten from a secondary using the old zone a mere millisecond before it updated.

And, you guessed it, spammers are still sending spam to the old MX.

(Since the new MX does spam tagging and the old one does not, this is vaguely irritating. If it was not a Friday, we might be doing something clever about the situation.)

I have to speculate about how the spam software behind this works. Clearly it doesn't do DNS lookups at the time it sends stuff, but does it do DNS lookups earlier and cache the results, or does it have a frontend that precomputes things all the way down to IP addresses? (The latter might be more useful, since it lets you use open relays too.)

Also, not all spammers and spam software does this; some spammers started hitting the new MX more or less the moment we published it, much faster than the places that send us legitimate email. (Which is not surprising; places that send us real email pretty much send us email regularly, which means that they have our MX entries in their DNS cache. A spammer's machine is probably not sending us email regularly, so is unlikely to have our MX already cached.)

Weekly spam summary on December 9th, 2006

Our SMTP frontend crashed and restarted three times this week, twice on Wednesday around 6pm and the third time today at 3:16pm, so some of our stats are really fragmentary. Still, this week we:

• got 15,036 messages from 272 different IP addresses.
• handled 20,984 sessions from 1,243 different IP addresses.
• received 114,833 connections from at least 33,061 different IP addresses up to Wednesday at 4am, received 92,925 connections from at least 27,804 different IP addresses from Wednesday at 6pm until Saturday at 4am, and received 11,972 connections from at least 5,107 different IP addresses since 3:16pm today.

This appears to make connection volume around the same as last week. For the days that we have decent per-day stats, connections are running around 38,000 to 40,000 connections a day, with around 10,000 to 11,000 different IP addresses added per day.

Kernel level packet filtering top ten:

Host/Mask           Packets   Bytes
213.29.7.0/24         28527   1712K
213.4.149.12          16263    846K
69.64.75.166          15633    938K
64.166.14.222         12648    607K
194.105.128.205        6435    386K
81.92.112.2            4551    273K
202.175.95.171         4357    261K
63.162.158.16          4176    200K
202.44.165.9           3892    187K
193.252.22.158         3480    209K

Things are up from last week overall.

• 213.29.7.0/24 is the centrum.cz mail servers, justifying their new permanent block.
• 213.4.149.12 is terra.es, returning from October and many, many previous appearances.
• 69.64.75.166 and 63.162.158.16 kept trying bad HELO greetings.
• 64.166.14.222 is still a PacBell DSL line.
• 194.105.128.205 and 81.92.112.2 both kept trying to send us stuff that had already tripped spamtraps.
• 202.175.95.171 is in the CBL.
• 202.44.165.9 has invalid reverse DNS and is in APNIC space; we require APNIC IP addresses to have valid reverse DNS.
• 193.252.22.158 is a wanadoo.co.uk machine, which has wound up being in SPEWS again and has appeared here before.

Connection time rejection stats:

  58045 total
  33557 dynamic IP
  19569 bad or no reverse DNS
   3319 class bl-cbl
    210 class bl-sdul
    190 class bl-dsbl
    104 class bl-spews
     89 class bl-njabl
     76 cuttingedgemedia.com
     42 class bl-ordb
     27 class bl-sbl

This week saw some really prolific connection time rejection sources. 13 of the top 30 most rejected IP addresses were rejected 100 times or more, with the champion being 125.246.18.130 (1,124 times, all in a few minutes around 6pm on December 3rd, with enough activity that it triggered our per IP address maximum connection limits). After that we drop to 64.166.14.222 (201 times), 63.138.101.140 (172 times), and so on.

In other stats, 22 of the top 30 are currently in the CBL, and 5 are currently in bl.spamcop.net.

Here's a table of the SBL hits:

Connections	SBL listing	comments
9	SBL45324	a /24 ROKSO listing for Expedite Media
5	SBL39631	a spam source in .cz (listed March 29th)
4	SBL47687	a spam source (listed October 27th)
2	SBL48728	reasonably long-term spam source
2	SBL48020	a /27 ROKSO listing for Howard Minsky (listed November 3rd)
1	SBL48694	a /24 of spammers
1	SBL48348	a /24 ROKSO listing for 'livemercial.com' (listed November 17th)
1	SBL46756	A ROKSO listing for William Stanley; apparently it is an open squid proxy being used by this spammer (listed September 18th)
1	SBL41737	a spammer's mail sending machine (listed May 10th)
1	SBL41344	a /21 listing for a spammer's web hosting (listed May 17th)

It's interesting that this time around there's not a single advance fee fraud spam source on the list. If I'm really lucky, this means that (SBL-listed) advance fee fraud spam sources are cleaning up their act, but I suspect that it is more likely that spammers are learning to not bother using SBL-listed free webmail systems.

This week, Hotmail had:

• 1 message accepted.
• No messages rejected because they came from non-Hotmail email addresses.
• 27 messages sent to our spamtraps.
• 1 message refused because its sender address had already hit our spamtraps.
• 3 messages refused due to their origin IP address (one from Nigeria, one from the Cote d'Ivoire, and one from Burkina Faso)

And the final numbers:

what	# this week	(distinct IPs)	# last week	(distinct IPs)
Bad HELOs	785	146	1059	155
Bad bounces	109	95	109	101

The first_last female Slavic names staged a huge comeback this week, although they don't quite make up a majority of the bad bounces; it looks like that honor is captured by the random alphanumeric jumble login names.

(One amusement in all of these stats is watching a single first name be associated with a run of last names, for example 'alisa petrova', then polkyakova, then osipova. I suspect that this is just brute force table merging of first names and last names.)

Weekly spam summary on December 2nd, 2006

Our SMTP frontend crashed and restarted today at 2:51pm, which means that some stats are a little bit distorted. This week, we:

• got 15,320 messages from 276 different IP addresses.
• handled 21,412 sessions from 1,467 different IP addresses.
• received 217,984 connections from at least 66,248 different IP addresses up until this morning at 4am, and 11,150 connections from at least 4,184 different IP addresses since 2:51pm.
• hit a highwater of 50 connections being checked at once by 4am this morning (and a less impressive highwater of 9 since 2:51pm).

Connection count is up from last week, although nothing else really is. Removing today from the per-day table, we have:

Day	Connections	different IPs
Sunday	40,151	+15,122
Monday	39,803	+12,027
Tuesday	31,702	+9,861
Wednesday	34,586	+10,595
Thursday	42,762	+10,402
Friday	28,980	+8,241

This is more see-sawing than we usually see, especially on Sunday. The highwater of 50 simultaneous connections was set on Thursday, which isn't too surprising.

Kernel level packet filtering top ten:

Host/Mask           Packets   Bytes
208.99.198.64/27      38955   2337K
213.29.7.0/24         29146   1749K
64.166.14.222         13032    625K
212.11.40.130          5965    358K
81.115.40.8            5039    269K
212.216.176.0/24       3996    199K
217.16.29.50           3975    239K
66.79.27.66            3896    234K
216.64.81.10           3454    166K
63.138.101.139         3369    162K

• 208.99.198.64/27 is SBL48200, returning from last week and now earning a place in our permanent blocks.
• 213.29.7.0/24 is centrum.cz, also returning and also earning a permanent block.
• 64.166.14.222 also returns from last week, still a PacBell DSL line. Evidently it really, really wants to talk to us.
• 212.11.40.130 and 81.115.40.8 are both generic 'dynamic' IPs, from easnet.fr and telecomitalia.it respectively.
• 217.16.29.50 aka by.ru spent too much trying to send us spam that had already hit our spamtraps.
• 216.64.81.10 kept trying a bad HELO too much.
• 63.138.101.139 is in the CBL. I note with interest that despite being called 'mx03.simon-mx.com', the netblock it is in allegedly belongs to 'IMARKETING CONSULTANTS' (under PaeTec), allegedly located in Florida.

Overall volume seems down from last week; there are fewer really active sources, discounting SBL48200.

Connection time rejection stats:

  70836 total
  45848 dynamic IP
  17887 bad or no reverse DNS
   5198 class bl-cbl
    645 class bl-sdul
    250 class bl-dsbl
     90 class bl-sbl
     61 class bl-njabl
     58 class bl-spews
     22 class bl-ordb

As I sometime like to say, yow! This may be the highest rejection count we've ever had, and it certainly seems like a significant spam storm hit us this week. The most active sources of dynamic IPs are:

   3064 rr.com
   2336 proxad.net
   1817 retail.telecomitalia.it
   1623 comcast.net
   1553 ono.com
   1423 dynamicip.rima-tde.net
   1383 user.auna.net
   1312 verizon
   1209 wanadoo.fr
   1118 charter.com

Only two of the top 30 most rejected IP addresses were rejected 100 times or more: 200.72.136.178 (135 times, rejected for being a LACNIC IP address with no reverse DNS) and our friend 63.138.101.138 (130 times). 21 of the top 30 are currently in the CBL and 9 are currently in bl.spamcop.net.

This week, Hotmail managed:

• 1 message accepted.
• 1 message rejected because it came from a non-Hotmail email address (in this case an address at 'alliedpersonelsvcinc.co.uk').
• 28 messages sent to our spamtraps.
• 2 messages refused because their sender addresses had already hit our spamtraps.
• 1 messages refused due to its origin IP address being in the Cote d'Ivoire.

This is better than last week, but that's still not saying very much.

what	# this week	(distinct IPs)	# last week	(distinct IPs)
Bad HELOs	1059	155	2148	161
Bad bounces	109	101	412	319

The clear winner in the bad HELO sweepstakes is 210.171.112.2, with 136 attempts before it got blocked. No one won the bad bounces sweepstakes; as you can guess from the numbers, only a very few places even sent us more than one.

This week the first_last login name pattern bounces went away almost completely. What's left is primarily plausible usernames (generally not ones that were ever valid here), leavened with a few alphanumeric jumbles.