Wandering Thoughts

2018-08-12

A recent spate of ZIP attachments with everything

Our program for logging email attachment type information looks inside .zip and .jar archives, including one level of nesting. Often what we see in this is routine, with basically the sort of content you'd expect from either routine stuff or malware, but recently we've been seeing zip archives that are just stuffed with at least one of almost any file extension you can think of. A few days ago we logged an extreme example:

1fnnAC-0003dZ-EP attachment application/zip; MIME file ext: .zip; zip exts: .jar; inner zip exts: .abc .abl .acc .ach .adc .adz[2] .afd .age .ago .agy .aht .ake .ala .alp .and .ans .aob[2] .aor .app .apt .ara .ary .aud .aus .ave .axe .baa .bag .bap .bat .bde .bet .bin .bis .bkg .boe .bra .bsh .buz .bye .cai .cal .cat .caw .cdg .chm .cit .class[10] .cli[2] .clo .col .cop .cpl .crc .crs .cst .ctg .cto .cup .cwt .dad .dbl .dcb .der .det[2] .dew .dey .dig .dil[3] .dks[2] .dur .dwt .dye .eft .ego .elb[2] .elm .els[2] .emf .emm[2] .emu .err .esd .esq .ext .eyn .fax .fbi[2] .fcs .fee .fei .fem .ffa .fgn .fig .flb .fly[3] .foe .fog .fud .gab .gae .gal .gas .geb .gig .gin .gio[2] .goa .gob .god .gon .goo .gox .gtc .gun .had[2] .hah .hak[2] .hao .hat .hau .hcb .hcl[2] .hed .heh .hen[2] .hes[3] .hia .hip .hir .hld .hoc .hoe .hts .hug .hye .ibo .ide .ihp[2] .ijo .ilk .imu .ing[2] .ipr[2] .iqs .ire .iwa .iyo[2] .jah .jap .jay .jct .jem[2] .jud .jur .kat .kaw .kay .key .khi .kop .kor .kos .kph .kyl .lab[3] .lap .lcm .lea .lek .les .lib .lid .lit .llb .lou .lub .lxx .mao .map .maw .meu .mf .mix .mks .mog .mor .mot .mph .mus .nee .nef .nei .nep .nut .oak[2] .obb .ofo .oki .one .oni .ops .ora .our .pan .pap .par .paw .pax .pay .pdq .peh .pep .pia .pie .pig .pit .pks .poh .pos .pot .ppa .pps .pre .pry[2] .psi .pwr .pyr .rab .ram .rat .raw .rct .ref .reg .res .rfs .rig .rim .rix .rld .roc .roi .rpm .rut .rux .rwd .rwy .rye .sab .sau .sds .sed[2] .sei .sel .sew .she .shr .sie .sil .sim .sip .six .sny .soe .sou .soy .sqq .stg .sum .sur[2] .syd .tar .tat .tay .ted .tef .tem .tng .ton .tou .twa .udo .uns .urb .urn .uti .vac[2] .vil .von .vum .wab .wae .wea .wop[2] .wot .wro[2] .wud .xii[2] .xiv .xxi .xxv .xxx .yam[2] .yay .yea .yeo .yer .yez .yoe .yrs .yun .zat .zen .zho .zig .zip .zod

(We deliberately log file extensions inside zip archives in alphabetical order, so it may well have had a much different order originally.)

This particular message was detected by Sophos PureMessage as 'Mal/DrodZp-A', which may be a relatively generic name. The Subject: of the message was the relatively generic 'Re: Invoice/Receipt', and I don't know what the overall MIME filename of the .zip was claimed to be. We've received a bunch of very similar attachments that were just .jars (not .zip in .jar) with giant lists of extensions. Many of them have been rejected for containing (nominal) bad file types, and their MIME filenames have been things like 'ORIGIAL SHIPPING DOCUMENTS.qrypted.jar' and "0042133704 _ PDF.jar".

(It's possible that these direct .jars would also be detected as Mal/DrodZp-A, but we reject for bad file types before we check for known viruses.)

I doubt that the attachment had genuine examples of these file types, especially things like .rpm (RPM packages) and .nef (Nikon camera RAWs, which are invariably anywhere from several megabytes to tens of megabytes for the latest high-resolution Nikon DSLRs). I'm sure that the malware has some reason for doing this spray of files and file extensions, but I have no idea what it might be. If there are some anti-virus products that give up if a .jar has enough different file extensions in it, that's kind of sad (among other things).

Sadly for any additional filtering we might considering doing, I suspect that the dangerous parts of this were in the actual Java stuff (eg the .class files) and everything else is distraction. It'd be somewhat interesting to pick through a captured sample, because I am curious about what's in all of those files (or if they're just zero-length ones put in to pad things out) and also what file names they have. Did the malware make up some jumble of random file names, or is it embedded a message in them or something clever? I'll never know, because it's not important enough to bother doing anything special for.

ZipAttachmentWithEverything written at 00:26:29; Add Comment

2018-07-27

Today I saw a spammer parasite itself on another spammer (probably)

We have an account request management system that sends out email to people as part of its activities, using an administrative email address as the sender address. We don't directly expose the address anywhere on our web pages, but it winds up in people's email address books when they get email from it and so years ago it leaked into the hands of spammers and we started to get occasional spam to it. Today it got two such pieces of email, both from and through Mailchimp and both theoretically sent by 'naturaful.com'.

The first one went like this:

From: Support Naturaful <support@naturaful.com>
Subject: INV04732 from Naturaful Support
Date: Thu, 26 Jul 2018 14:49:59 +0000

View invoice ([an-URL])
$ 1750.00 due 30 July
INV04732

[...]

The URL went off to a random and likely hijacked URL on a random website, or at least it tried to; it was probably broken (one part of it was a literal '[UNIQID]' as a query parameter). This was clearly basically a phish spam, and it appears to have tried to redirect from the initial URL to an invoice page on 'xerotransfers.com', where it would presumably have tried to extract some sort of payment from visitors.

It was followed less than two hours later by a second email message, a rather flustered one:

From: Support Naturaful <support@naturaful.com>
Subject: We're Sorry - Please Ignore Email About Invoice
Date: Thu, 26 Jul 2018 16:35:10 +0000

Please ignore the last email about a large invoice amount. .
Please do not click on the button or pay any money.
Any links that do not have [list of domains] is not our website. Any sales of Naturaful products are paid on our website and you don't owe anything after.
Please ignore the last email, we're currently cleaning up our database and ensuring this does not happen again.
Security is our primary concern.

What appears to have happened here is that our administrative address was bought by naturaful.com and added to a mailing list that they were going to use to send out spam (through Mailchimp). Before they could use their shiny new mailing list to send out their own spam, another spammer came by and exploited a security vulnerability of some sort to hijack naturaful.com's mailing list and Mailchimp account (and 'good' name) to send out their own spam.

As a bonus prize, naturaful.com claims to be in Canada, which makes what they're doing completely unambiguously illegal under our anti-spam law. The odds are that the government will never get around to doing anything to them, but one can always hope. In the mean time, neither these people nor Mailchimp are going to be successfully sending email to this particular administrative address.

(As far as Mailchimp goes, well, they know what business they're in and they're evidently not interested in doing better even though they certainly could.)

(This elaborates on my tweet.)

SpammerOnSpammer written at 02:16:11; Add Comment

2018-07-23

The irritatingly many executable formats of Windows

So I tweeted:

It's impressive how many different executable file formats Windows has.

(I care because our email system wants to reject top-level attachments that are Windows 'executables' and boy is the list getting long.)

I put 'executables' into quotes in this tweet because many of these file formats (or more exactly file types) are not binaries; instead they're text files that Windows will feed to various things that will interpret them in ways that you don't want. Typical extensions that we see as top level attachments (and reject at SMTP time) include .lnk, .js, .bat, .com, .exe, .vbs, and .vbe. Some of these are encoded binaries, while others are text.

We mostly do this checking and rejection based on MIME file extensions, partly because it's easiest. Also, for the ones that are text (and at least some of the ones that are encoded binaries), my understanding is that what makes them dangerous on a Windows machine is their file extension. A suitable text file with the extension ".txt" will be opened harmlessly in some editor, while the same file with the extension ".js" will generally be run if you try to open it.

(We do some file content sniffing to look for and reject unlabeled Windows executables, ie things which libmagic will report as type 'application/x-dosexec'. As you can see here, there are actually a lot of (sub)formats that map to this.)

We've historically added extensions one at a time as we run into them, usually when our commercial anti-spam system rejects one of them as being a virus (this time, several .pif files being rejected as 'W32/Mytob-C'). Possibly this is the wrong approach and we should find a master list somewhere to get almost all of this over with at once (perhaps starting from GMail's list of blocked file types). On the other hand, there's some benefit to passing up rejections, especially if you don't actually seem to need them. If we never see file types, well, why block them?

(I'm not completely convinced by this logic, by the way. But I'm lazy and also very aware that I could spend all my time building intricate anti-spam precautions of dubious actual benefit.)

WindowsManyExecutables written at 00:45:13; Add Comment

2018-06-10

People receiving email don't feel it should be their job to stop spam

There is a popular observation when people get spam, especially spam from places that specialize in sending email such as Sendgrid, and that is that these places are not infrequently pretty good about handling spam issues. Sometimes they have 'complain that this is spam' links right in the email headers (not Sendgrid, though), or similar things. But thinking that this is sufficient is a mistake, perhaps not in the short term but definitely in the long term (in my opinion).

What places that are merely 'good at responding to complaints' are really doing is making it the job of people receiving email to deal with spam. Unsurprisingly, not infrequently people are unhappy with this, either quietly or loudly. As a general rule, people do not want it to be their job to deal with spam; they want it to be your job (where 'you' is some combination of the people operating their mail system and the people responsible for sending email to them). That in practice it currently is their job, requiring a bunch of labour and a bunch of annoying workarounds and hacks, is something that people are not exactly pleased about. They consider this a flaw in the system, and it is.

(Spam in general is a fundamental hard problem, but spam from major mail sending providers is merely because they aren't interested in trying hard enough to stop it.)

Of course, people receiving email do not necessarily understand the situation in the way I've put it here. But we do know that they are unhappy to get spam (there are very few people who are happy to get it); even if they are 'happy' to be able to do something to make it go away, they would be happier to not have to deal with spam at all. The more work they have to do and the less certain that work is in making the spam go away, the more unhappy they are, until you reach the stage of someone like me.

PS: A great many things are 'making it the job of the receiver to deal with spam', including the very basic one of 'I think I will use GMail and live with its drawbacks because they have really good spam filtering'. Certainly not taking genuinely effective steps to stop spam senders and then waiting for the complaints to roll in is one of them; the mail sending place has effectively outsourced a significant part of its spam detection to the people receiving its email. If said people did nothing, the mail sending place would not know it was being used to send spam.

PPS: To cut off one possible reply in advance, a claim of 'if no one complains, clearly it wasn't spam' is demonstrably and obviously false and anyone advancing it as a defense is either morally vacuous or doing so in a cynical manipulation to deflect criticism. That marketing and advertising organizations love to make claims like this is one reason why a great many people hate them.

ReceiversStopSpamNotJob written at 23:24:10; Add Comment

2018-06-09

How to run a mail sending service that will probably never send spam

I have written any number of times before that mail sending services could take steps that would make sure almost no spam would be sent through them, but they don't bother (eg on Amazon, on modern mailing list services in general and earlier). However, I have not written down my view of these steps, partly because I have considered them obvious in the community in general. For various reasons, I now feel like writing these steps down. So here is an incomplete list of obvious steps to take that would mostly gut sending spam through such a service.

In no particular order:

  • Charge people a decent amount of money for your service, possibly with a deposit up front. Don't have a free or a cheap tier, because it attracts the wrong sort of customer (Patrick McKenzie has written at length on how too-low or free pricing is a bad idea in SaaS in general).

  • Force people to put their address lists on your service, not just funnel their email through in bulk sending. Forced uploads allow you to scan the address list in advance to look for known warning signs, such as definitely nonexistent domains or known-bad addresses that never accept your email.

  • Require all email addresses submitted to you by a particular customer to be confirmed. The gold standard would be confirming separately for every alleged mailing list the customer sets up; the silver standard is confirming once when the customer first submits the address as part of any list and then assuming that the customer has the right to use that address in other lists. As part of requiring confirmation, provide an extra link that communicates to you 'I have never heard of these people and I do not know why they have my email address'. Even a moderate level of use of this link is a warning sign.

    It should go without saying that having more than a trace level of bounces or email rejections during confirmation should be a big warning sign.

    Probably the silver version is the most realistic, since these days customers may not have distinct 'mailing lists' as such, if they're using you to deliver event-based notifications to people's email and so on.

    (Even sending an initial notification email to people saying 'your address has been added to our system by <customer>' would be a step forward. These days a mail sending service could claim it was a GDPR requirement.)

  • Run all submitted mailing list messages through all of the available free open source anti-spam and anti-virus systems, and perhaps at least one of the commercial ones. If any of the systems flag the message, don't send the email and surface this in an alert both to the customer and to your abuse handling team.

    (It's not a service to the customer to let them send out email that you know will trip spam alerts for some recipients. Legitimate customers will likely thank you for such a pre-check service, and may even want a way to submit draft messages to it.)

  • Make it trivial for people to report unsolicited email and spam, and to 'unsubscribe'.
  • Pay attention to bounces, SMTP rejections in general, unsubscribes, and spam complaints. Mine them for addresses to add to your list of warning addresses. Rejections after SMTP DATA are probably an especially bad warning sign, because they suggest it was content filtering that caused the rejection.

    (As part of this, you should obviously recognize and parse the various SMTP 4xx and 5xx messages that major email providers use when they're dealing with questionable email messages. But this is so obvious that I suspect that any mail sending SaaS that wants to be successful is already doing it.)

I'm assuming as a baseline that you will do things like accept bounces and replies and properly implementing SMTP. These days you may want SPF, DKIM, or DMARC in order to pacify various large email providers who are getting increasingly insistent on it, but that's more in the realm of 'competently operating a commercial service'.

I'm pretty confident that any mail sending service that implemented all of these would send almost no spam, and I'm reasonably confident that it would still have a business. But of course it wouldn't have a business that's as big as you'd get by not bothering to do some of these things (especially confirming email addresses), and it would cost more to operate, and you wouldn't have as many customers because a certain number of the more shady people would stay away (as would all of the cheap people).

GoodMailSendingHygiene written at 00:27:22; Add Comment

2018-05-13

My GDPR pessimism

The latest great hope of various people, more or less including myself, is that the European GDPR will come along and put an end to various sorts of annoying email marketing activities and other intrusive ad and marketing activities. Under the GDPR, so goes the theory, companies like Yubico and Red Hat will not be able to abuse email addresses they happen to have sitting around to send marketing email; in fact they may not even have those email addresses sitting around at all.

(At least for people in the EU. The further great hope of the GDPR is that many companies affected by it won't bother trying to tackle the near-impossible task of figuring out who's in the EU and who's not.)

I'd like to believe this, but I'm not sure that I do. I'm not basing this on any examination of the GDPR or on what people have written about it. Instead, my pessimism comes from the cynical version of the Golden Rule. My simple observation that regardless of what they say, governments very rarely actually kill off entire decent-sized industries and slap large fines on a wide variety of prosperous and perfectly normal corporations who are conducting business as usual. It might happen, but it seems much more likely that there will be delays and 'clarifications' and so on that in the end cause the GDPR to have little practical effect on this sort of activity. If there is change, I expect it to happen only very slowly, as countries slow-walk things like fines as much as possible in favour of 'consulting' and 'advising' and so on with companies.

(In other words, a lot of stern letters and not many other effects. And I expect companies to take advantage of this to stall as much as possible, and to plead implementation difficulties and other things that tragically mean they can't comply quite yet. It may all be very theatrical, in the 'security theater' sense.)

Partly I come by this pessimism by watching what's happened with Canada's theoretically relatively strong anti-spam law. One of the strong features of this law was that it created a private right of action, where you could start a civil case against violators and thus you didn't have to depend on the official regulator getting around to doing something. Since Canada is a loser-pays legal system, this was always going to be a reasonably risky activity, but then in 2017 this provision was quietly suspended, including the charming quote:

The Government supports a balanced approach that protects the interests of consumers while eliminating any unintended consequences for organizations that have legitimate reasons for communicating electronically with Canadians.

This provision has yet to be revived, and there have been no 2018 enforcement actions by the CRTC under CASL (at least none that appear in the CRTC's public records).

It's possible that the EU will be more aggressive and determined about the GDPR and violations of it than Canada has been about our lauded (at the time) anti-spam law, especially in today's climate with increased public concern about these sort of issues, but I'm not going to hold my breath.

PS: It turns out that there has been some activity on the CASL front (and, and, and, and) and there may be good news someday. But if so, it will probably be significantly later than the already slow timeline that CASL itself specified. Applications to the speed of GDPR are left as an exercise for the reader.

GDPRPessimism written at 22:08:13; Add Comment

2018-05-09

Yubico fails to care that people give you email addresses for specific purposes

A while back, Yubico had a little security issue that forced it to replace any number of Yubikey 4s, including mine. In order to do this, they required people to give them an email address so they could send you some necessary information; following my usual practice I gave them a tagged, individualized address. Today I received email to that address, received from the server of a domain called 'mktomail.com', that started out like this:

Subject: Passwordless authentication is here

Yubico scales across enterprise

Passwords are out. You're in!

The passswordless evolution of the FIDO U2F standard has arrived with FIDO2. [... marketing materials removed with prejudice ...]

You are receiving this email because you made a Yubico purchase or contacted Yubico.

I'm sorry, that's not how this works. In the normal course of events, people do not give you email addresses to do with as you will; people give you email addresses for specific purposes. In this case, I gave Yubico an email address to get a defective product fixed, but one might report a bug, contact product support, or perform other limited interactions with the company. These specific and limited purposes do not include 'receive unsolicited commercial marketing emails'.

Of course, the marketing department does not want to hear this. The marketing department wants to use every plausible address it can get its hands on. People these days vaguely get that you usually cannot buy addresses from other people without getting badly burned, but they keep thinking that other addresses are fair game, regardless of the purpose for which they were originally handed to the company.

Some of the time, the company supports the marketing department, as it did at Yubico, and these addresses get used outside of the purpose they were given to the company. At that point the company betrays the trust of the people who handed over their email addresses in good faith and pisses off some number of people who have interacted with the company in the past, some of which have actually bought their products. The results are predictable, as is the resulting form-letter evasion.

(When enough companies do this sort of thing for long enough, you get things like the EU's GDPR, which will likely make this conduct illegal. Sadly it is probably not illegal under Canada's anti-spam legislation, and anyway I expect Yubico to ignore the GDPR issues until they or someone else visible gets slapped with a nice fine for this sort of thing.)

Sadly I have no idea what is a viable alternative to Yubikeys, but at least we're not likely to buy any more any time soon.

AddressesLimitedPurposes written at 02:41:48; Add Comment

2018-04-28

What sorts of good email attachments our users get (April 2018 edition)

I've looked at various breakdowns of bad attachment types that get sent to our users, but of course that's not the only reason we collect all of this data. In fact it's the lesser reason; the greater one is to know the legitimate types of files our users get in email. So today I'm going to look at a week's worth of data from our central mail server, which is logged after all rejecting, filtering, and spam tagging has been applied.

Over that week, we logged 4,166 attachments from 3,093 email messages. Some email messages had quite a lot of attachments; the winner had 26 attachments, and then there's one with 23, two with 12, four with 9, nine with 8, and I've run out of patience to count from there. The median message has one attachment, though, as you'd expect.

Almost all of the attachments had MIME filenames; only 50 didn't. For those 50, the MIME types varied, with the most popular one being message/rfc822, but there are also images, text/plain, text/html, PGP signatures, and apparently one Office XML file. For the attachments with MIME file extensions, the most popular types break down like this:

  2339 MIME file ext: .pdf
   429 MIME file ext: .docx
   273 MIME file ext: .jpg
   159 MIME file ext: .xlsx
   125 MIME file ext: .png
    91 MIME file ext: .doc
    72 MIME file ext: .ics
    69 MIME file ext: .txt
    57 MIME file ext: .asc
    54 MIME file ext: .html

There were 72 different MIME file extensions in total, although some of them are clearly not real file extensions but instead just parts of the filename that happened to go after a dot. One is a timestamp, for example. These may be for regular filenames that had extra stuff added on, for example 'file.pdf.<timestamp>'.

The popularity of PDF files is no surprise, given that we're a university department. That may also explain how MS Word scores relatively highly (and perhaps the spreadsheets too, but I don't know there). All of the .asc cases are PGP signatures (and were sent with the MIME type application/pgp-signature), and some of them come from mailing list email that I get.

I took a look at MIME type information, and unsurprisingly it is somewhat less reliable than MIME file extensions. For instance, here is the MIME type breakdown for .pdf attachments:

  2160 application/pdf
   175 application/octet-stream
     1 pdf
     1 application/octetstream
     1 application/octet
     1 application/download

Looking at all attachments, application/octet-stream was the third most popular MIME type. Mostly it's used for PDFs, but there is a long tail of MIME filename extensions, which doesn't really surprise me. If a mail program is attaching something to a message and it's not completely sure what it is, application/octet-stream will get the job done and no one can really argue with you for picking it.

(Sometimes I look at this data and what I find is, well, data.)

GoodAttachmentTypes-2018-04 written at 01:19:47; Add Comment

2018-04-20

Spam from Yahoo Groups has quietly disappeared

Over the years I have written several times about what was, at the time, an ongoing serious and long-term spam problem with email from Yahoo Groups. Not only was spam almost all of the Groups email that we got, but it was also clear that Yahoo Groups was allowing spammers to create their own mailing lists. I was coincidentally reminded of this history recently, so I wondered how things were today.

One answer is that spam from Yahoo Groups has disappeared. Oh, it's not completely and utterly gone; we rejected one probable spam in last December and two at the end of July 2017, which is almost as far back as our readily accessible logs go (they stretch back to June 15th, 2017). But for pretty much anyone, much less what it was before, that counts as completely vanished. Certainly it counts for not having any sort of spam problem.

But this is the tip of the iceberg, because it turns out that email volume from Yahoo Groups has fallen off significantly as well. We almost always get under ten accepted messages a day from Yahoo Groups, and some days we get none. Even after removing the spam, this is nothing like four years ago in 2014, when my entry implies that we got about 22 non-spam messages a day from Yahoo Groups.

At one level I'm not surprised. Yahoo has been visibly and loudly dying for quite a while now, so I bet that a lot of people and groups have moved away from Yahoo Groups. If you had an active group that you cared about, it was clearly time to find alternate hosting quite some time ago and probably many people did (likely with Google Groups). At another level, I'm a bit surprised that it's this dramatic a shift. I would have expected plenty of people and groups to stick around until the very end, out of either inertia or ignorance. Perhaps Yahoo Groups service got so bad and so unreliable that even people who don't pay attention to computer news noticed that there was some problem.

On the other hand there's another metric, the amount of email from Yahoo Groups that was rejected due to bad destination addresses here (and how many different addresses there are). We almost always seen a small number of such rejections a day, and the evidence suggests that almost all of them are for the same few addresses. There are old, obsolete addresses here that have been rejecting Yahoo Groups email since last June, and Yahoo Groups is still trying to send email to them. Apparently they don't even handle locally generated bounces, never mind bounces that they refuse to accept back. I can't say I'm too surprised.

Given all of this I can't say I regret the slow motion demise of Yahoo Groups. At this point I'm not going to wish it was happening faster, because it's no longer causing us problems (and clearly hasn't been for more than half a year), but it's also clearly still not healthy. It's just that either the spammers abandoned it too or they finally got thrown off. (Perhaps a combination of both.)

YahooGroupsDisappeared written at 01:44:09; Add Comment

2018-03-30

Sometimes, not trying to reject some sort of spam is the right answer

I've written before about not doing anything about a temporary spate of spam, and it remains a useful guideline. But sometimes you're pretty convinced that certain spam patterns are long-standing, and it turns out that the right answer is still to not do anything, however reluctantly. As it happens, I have an example that we recently decided on.

One of the patterns we observe is that a decent amount of the attachments we get come from IPs listed in the Spamhaus Zen DNSBL. A further pattern we've seen is that a decent amount of those are detected as malware (see eg this), and we've also seen that there are some highly active Zen-listed sources (see this set of numbers from January). Given all of this, I recently put forward the idea of rejecting all messages from Zen-listed IPs that had an attachment, for the same broad reason that we reject some sorts of attachments; we're almost completely sure that these emails are bad and they're often dangerous, but our commercial anti-spam package may not pick the malware up on its own and cause us to reject them.

When I put it that way, this probably sounds good, and certainly that's how I thought of the idea when I proposed it. Then I put together some numbers, based on how many messages we would actually be shielding users from if we did this. It turned out that many of the messages were already being rejected and almost all of the remaining messages were already being scored as spam (and when I say 'almost all', I mean 816 out of 820).

We had a long discussion and decided that we weren't going to reject these messages. There are local reasons for why not that I'm not going to get into, but apart from them there is a larger one that caused me to not argue too hard for the rejections, which is that this doesn't seem like something with a high payoff in practice. It's not just that the volume is not huge; it's also that basically everything is already being detected as bad (and at least some of our users are discarding the email based on that).

There's an almost infinite set of things that you could do to reduce spam, with some payoff (and many with a reasonably worthwhile one). The challenge about anti-spam work is not finding things to do to reduce spam, it is partly in not doing things, because every thing you do has a cost that goes with its benefits. Sometimes that cost is too high relative to the gain, and it's not because the particular sort of spam is temporary; it's because the sort of spam is already being blocked well enough as it is, even though you could do better.

Sure, some of our users could ignore the 'this is probably spam' warnings and fall for malware that we allowed to be delivered to them. There could even be bad stuff in those four email messages that weren't scored as spam (to be honest, there probably was at least spam). But our existing system is doing well enough even though it's not perfect, and it's already complicated enough. So doing nothing this time is the right answer.

(It helps here that in the past I've enthusiastically put in some clever anti-spam trick, only to have it make somewhat less impact than I was hoping for. That's not a good feeling either.)

PassingUpSpamRejections written at 01:44:00; Add Comment

(Previous 10 or go back to March 2018 at 2018/03/28)

Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.