Wandering Thoughts archives

Looking at whether Zen-listed IPs keep trying to send us email

Here's a question: when an IP address listed in the Spamhaus Zen gets rejected, does it come back later or are most visits a one-time thing? This time I pulled 90 days worth of logs, extracted each day's rejections from Zen-listed IPs, and checked to see how many IPs showed up in more than one day's logs.

(Because an IP could be trying to deliver stuff right when the logs roll, the safe question is how many IPs show up in more than two days worth of logs.)

The first answer is that we have some persistent IPs but not anything that is really hammering on us. Well, at least if you look at the data this way. Here, have a table:

212.174.85.130	24 days	SBL107558
89.204.63.228	20 days	SBL168886 and the PBL
189.112.34.215	18 days	SBL153384
82.165.159.34	15 days	web.de; SBL175032
82.165.159.35	13 days	web.de and SBL175032 again
82.165.159.3	10 days	web.de but now SBL175030, which is basically the same as SBL175032; web.de is clearly good at getting SBL-listed.
217.133.203.34	10 days	SBL157999
115.93.88.50	10 days	In the PBL
82.165.159.2	9 days	web.de yet again, SBL175030
218.38.136.79	9 days	SBL146938
216.104.35.85 216.104.35.86 216.104.35.90	9 days	No longer listed.
200.68.99.196	9 days	SBL CSS
186.1.192.23	9 days	SBL172432

(This table probably doesn't look that nice in the syndication feed.)

Now things get interesting, because I noticed a pattern and went digging. All of the IPs from 216.104.35.83 through 216.104.35.94 got rejected by us at various times in the 90 days, and all of them were rejected on multiple days. Even more interesting, the rejections stretch from day 11 through day 90 (although not continuously).

(The gaps in rejections could be either because they stopped sending to email addresses that were rejecting them, because they dropped out of Zen temporarily, or both of the above.)

This prompted me to look at /24-based reoccurrence, and there things get more interesting:

173.242.121.0/24	46 days	One IP still in the SBL CSS
198.64.159.0/24	45 days	13 of 23 IPs still in the SBL CSS
216.104.35.0/24	43 days	Nothing still listed out of the 12 IPs we rejected from this
82.165.159.0/24	30 days	web.de, mentioned above; all four IPs still in their SBL listings
177.47.102.0/24	27 days	SBL136747, a /24 listing dating from August 14, 2012
212.174.85.0/24	26 days	SBL107558; one of the single IPs made it into the single-IP list
178.210.168.0/24	25 days	Multiple IPs still in the SBL CSS
216.229.59.0/24	22 days	Multiple IPs still in the SBL CSS

I'm going to stop here because the next '/24' is actually due to a single IP (89.204.63.228) so we're reaching the crossover point (besides, I'm doing this all more or less by hand).

What really surprises me from looking at the by-/24 breakdown is how active the SBL CSS clearly is. If someone told me that the SBL CSS was now the largest single contributor for spam rejections, I wouldn't be surprised.

(I can't verify that without changing our mail configuration to add more logging (since SBL CSS listings expire, we'd have to capture the Zen results at the time of the actual rejection). Sadly my curiosity is not worth that.)

(This is kind of a followup to looking to see if IP addresses persist in Zen.)

Sidebar: a way in which these results may not be representative

We do Zen-based rejections only for some email addresses (only those that have opted in to it). So a Zen-listed sending IP wouldn't necessarily see continuous rejections if they kept sending to us. It depends on what email addresses they are sending to that day and they could have a day with no rejections.

I haven't tried to dig into the raw logs to see if this is happening for some of these IPs, or in general if these IPs saw a mix of successful deliveries and rejections or if they saw uniform rejections. I don't know if I'll ever do this level of analysis, since it's going past what I can easily bash together with shell scripts and awk. Past the land of shell scripts lies the land of real work.

Looking at whether (some) IP addresses persist in zen.spamhaus.org

After writing my entry on the shifting SBL I started to wonder how many IP addresses we reject for being SBL listed stop being SBL listed after a (moderate) while. I can't answer that directly, because we actually use the combined Zen Spamhaus list and we don't log the specific return codes, but I can answer a related question: how many Zen-listed IP addresses seem to stay in the Zen lists?

To check this, I pulled 10 days of records from January 18th through January 27th, extracted all of the distinct IPs that we found listed in zen.spamhaus.org, and re-queried Zen now to see how many of them are still there. Over that ten day period we had 613 Zen-listed IP addresses; today, 534 of them are still in the Zen. So a fairly decent number stay present for 30 days or more.

(Technically some of them could have disappeared and then reappeared.)

I also pulled specific return codes for all of those IP addresses, so I can now give you a breakdown of why those 534 addresses are still present:

• 420 of them are in Spamhaus-maintained PBL data. There's no single really big source, but 46 of them are from Beltelecom in Belarus (AS6697) and 23 are from Chinanet (AS4134).

• 70 of them are in the XBL, specifically in the CBL.

• 56 are in the SBL. There's no really big source, but five IPs are from 177.47.102.0/24 aka SBL136747, four are from 5.135.106.0/27 aka SBL173923, and two are from 212.174.85.0/24 aka SBL107558.

  (Two of those SBL listings are depressingly old, not that I am really surprised by long-term SBL listings by this point.)

• 47 of them are in ISP-maintained PBL data.
• 9 of them are in the SBL CSS, which is pretty impressive and depressing because SBL CSS listings expire fairly fast.

An equally interesting question is how many of those 79 now-unlisted IPs are listed in some other DNS blocklist. The answer turns out to be a fair number; 60 are still listed on some DNS blocklist that I have in my program to check IPs against a big collection of DNSBls. Many but not all of the hits are for b.barracudacentral.org (which is not a DNSBl that I consider to be really high quality; it seems to be more of a hair-trigger lister).

(I'm out of touch with what's considered a high-quality DNSBl versus lower-quality ones so I'm not going to offer further reporting or opinions.)