Wandering Thoughts archives

2018-03-30

Sometimes, not trying to reject some sort of spam is the right answer

I've written before about not doing anything about a temporary spate of spam, and it remains a useful guideline. But sometimes you're pretty convinced that certain spam patterns are long-standing, and it turns out that the right answer is still to not do anything, however reluctantly. As it happens, I have an example that we recently decided on.

One of the patterns we observe is that a decent amount of the attachments we get come from IPs listed in the Spamhaus Zen DNSBL. A further pattern we've seen is that a decent amount of those are detected as malware (see eg this), and we've also seen that there are some highly active Zen-listed sources (see this set of numbers from January). Given all of this, I recently put forward the idea of rejecting all messages from Zen-listed IPs that had an attachment, for the same broad reason that we reject some sorts of attachments; we're almost completely sure that these emails are bad and they're often dangerous, but our commercial anti-spam package may not pick the malware up on its own and cause us to reject them.

When I put it that way, this probably sounds good, and certainly that's how I thought of the idea when I proposed it. Then I put together some numbers, based on how many messages we would actually be shielding users from if we did this. It turned out that many of the messages were already being rejected and almost all of the remaining messages were already being scored as spam (and when I say 'almost all', I mean 816 out of 820).

We had a long discussion and decided that we weren't going to reject these messages. There are local reasons for why not that I'm not going to get into, but apart from them there is a larger one that caused me to not argue too hard for the rejections, which is that this doesn't seem like something with a high payoff in practice. It's not just that the volume is not huge; it's also that basically everything is already being detected as bad (and at least some of our users are discarding the email based on that).

There's an almost infinite set of things that you could do to reduce spam, with some payoff (and many with a reasonably worthwhile one). The challenge about anti-spam work is not finding things to do to reduce spam, it is partly in not doing things, because every thing you do has a cost that goes with its benefits. Sometimes that cost is too high relative to the gain, and it's not because the particular sort of spam is temporary; it's because the sort of spam is already being blocked well enough as it is, even though you could do better.

Sure, some of our users could ignore the 'this is probably spam' warnings and fall for malware that we allowed to be delivered to them. There could even be bad stuff in those four email messages that weren't scored as spam (to be honest, there probably was at least spam). But our existing system is doing well enough even though it's not perfect, and it's already complicated enough. So doing nothing this time is the right answer.

(It helps here that in the past I've enthusiastically put in some clever anti-spam trick, only to have it make somewhat less impact than I was hoping for. That's not a good feeling either.)

PassingUpSpamRejections written at 01:44:00; Add Comment

2018-03-28

The correlation between Spamhaus Zen listings and attachment types (March 2018 edition)

Our program to capture information about what sort of email attachments our users get logs not just the attachment information but also whether or not the sending IP address was listed in zen.spamhaus.org at the time. For reasons beyond the scope of this entry, today I want to look at the correlation between sending us attachments and being in Spamhaus Zen, and what attachment types are popular. Because it's the most convenient option, I'm going to use four weeks of recent logs.

Over this time we logged 15,900 incoming messages with attachments (20,395 attachments total), although given my previous experience it's possible that some of these are repeated attempts from misbehaving senders that believe permanent SMTP rejections are just temporary. 3,890 of these messages (3,925 attachments total) were in the Spamhaus Zen at the time, or about one quarter, which is neither huge nor insignificant. The most popular attachment types for Zen listed IPs to send us are as follows:

  1312 MIME file ext: .html     [89%]
   754 MIME file ext: .docx     [38%]
   516 MIME file ext: .doc      [59%]
   352 MIME file ext: .xlsx     [51%]
   120 MIME file ext: .xls      [54%]
    85 MIME file ext: .pdf      [ 1%]
    57 MIME file ext: .pdf.gz   [87%]
    42 MIME file ext: .ace      [28%]

(For simplicity I'm looking only at things with MIME file extensions. 705 attachments in total, 435 from Zen-listed IPs, did not have file extensions. Almost all of the Zen-listed ones were Microsoft Word documents, usually .docx.)

The percentages are against the total number of that attachment type we received. PDFs by far our most popular attachment type in general, followed by .docx, .html (almost all from Zen listed IPs), JPGs, .doc, .png, and .xlsx.

The '.pdf.gz' attachments are actually all .exe files in disguise, which we reject. I'm not sure why malware tries this, but presumably it works on some people and some systems. The .html attachments are very likely to be what our commercial spam filtering system scores as 'Mal/Phish', because this is a pattern we see all the time. We reject all .ace attachments in general as they're all malware, so I find it interesting that only 30% of them come from Zen listed IPs; there seem to be a fair number of malware senders that aren't in Zen.

(This is something to bear in mind if you feel that the Zen alone will do a great job of protecting you. I'm not saying it won't; it depends on what you're worried about and what the attack patterns are against you.)

With the exception of .html (and the special case of .pdf.gz), there's no attachment type that is clearly beloved of Zen-listed IPs, although they're over-represented in a number of them (where they're roughly half of those attachment types despite being only roughly a quarter of our attachment volume). This seems mostly likely to be due to relatively low usage by legitimate senders rather than high usage by Zen-listed IPs. In turn this is probably because of the profile of our users probably tilts away from the use of Microsoft Office files.

Because of how we do server side spam filtering, some amount of Zen-listed IP addresses that would send us attachments don't make it this far, because they get entirely rejected at RCPT TO time. It's difficult to estimate how many such rejections we might have, so I'm not going to guess or try to throw raw numbers around.

PS: If I'm understanding the logs correctly, the number of Zen-listed IPs that sent us attachments is a drop in the bucket compared to the total number of Zen-listed IPs that got as far as submitting email. My log analysis suggests that there were roughly 77,500 such email submissions over the same time period, from 18,800 different IPs.

(See also the related attachment types we see in email from Zen-listed IP addresses, from last December. Some of the patterns have clearly shifted since then. For the absolute numbers, note that I did nine weeks of data then and I'm doing four weeks now.)

Sidebar: Some people are confused about MIME types

We got two messages with JPGs that were attached with the MIME type '*/*', which is not how MIME types work. Sadly I suspect that many mail clients will display the JPGs anyway, because that's how the Internet works (and it's not necessarily a bad thing, and even when it is it's hard to persuade people of that).

(Someone may have been thinking of browsers when they generated that MIME type, or they may just have been refusing to even try.)

ZenAndAttachments-2018-03 written at 02:22:15; Add Comment

2018-03-12

A spammer misses a glorious opportunity

Most of the spam that I collect on the machines that I run my sinkhole SMTP server on is boring spam. Since it's boring, I've tried to block as much of it as possible; still, there are plenty of cases that get through, because that sort of spam can come from all over. Today I got what initially looked like one of those boring spams that sneak through. It appeared in my log like this:

[...] from 196.15.235.161 / <REDACTED@justice.gov.za> to <REDACTED>: [...] helo 'mail3.justice.gov.za' [...]

I saw that and shrugged; clearly it was another forged advance fee fraud spam, just like the ones claiming to be from the FBI. But when I looked at the full metadata of the logged message, I got a surprise. There in the metadata was the resolved, verified DNS name of the sending IP and it was mail3.justice.gov.za. This wasn't email pretending to be from the South Africa's Department of Justice; this actually was email from one of the DoJ's mail servers. The reverse DNS is real and valid, and in fact this IP is one of the four MX servers for justice.gov.za (a second MX server is right beside it in that /24).

So why do I call this a spammer missing a glorious opportunity? Well, let me show you the important bits of the spam message itself:

From: REDACTED <REDACTED@justice.gov.za>
To: "info@cc.com" <info@cc.com>
Subject: HELPDESK

To All,

Today Monday 12th of March 2018. We are shutting down your present web-mail to create space for 2018 Outlook Web Access with a high visual definition and Space.
This service creates more space and easy access to email. Please update your account by clicking on the link below and fill information for Activation.

[...]

That's right. Given the golden opportunity of access to the real, legitimate mail servers of the Department of Justice of South Africa (likely via a compromised account), the spammer used it to send not the most genuine looking advance fee fraud you could imagine, but a garden variety, completely untargeted phish spam.

Of course there's decent, boring reasons for this. For a start, the actual IP address source of advance fee fraud spam is completely unimportant, because the recipients who will even think of checking that aren't the kind of people who will fall for the spam in the first place. If anything, advance fee fraud spammers apparently may deliberately make their spam look bad and suspicious, so that anyone who actually answers is highly likely to be gullible enough to go through with the whole thing, instead of wasting their time. If that's so, sending from the real justice.gov.za is, if anything, a thing to avoid.

Still, I wish the spam message had been advance fee fraud. That's the way the universe should be when you get the chance to use justice.gov.za for your spam.

SpammerMissedOpportunity written at 22:54:33; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.