The various meanings of DKIM signing message headers
When I talked about the issue of what headers to include in email DKIM signatures, I didn't really cover the specifics of how you DKIM sign email headers and what the various options mean. The specifics can matter, especially since they help you (me) understand and navigate through the options that mailers (such as Exim) offer here.
In email messages, DKIM signatures appear in a
header, which lists a bunch of parameters:
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=list.zfsonlinux.org; h=from:to:subject:message-id:in-reply-to:references:date [....]
The 'h=' list (which isn't complete here) is a list of headers that have been signed. More specifically, it's a list of instances of headers. If there are multiple instances of a given header in a message, DKIM defines an order to them and the instances of the header are checked (or used) in that order. So if you include 'from' once in the DKIM header list, you are saying that your DKIM signature includes DKIM's first 'From:' header in the message. If a second 'From:' header is added to the message, it's not included what's covered by your DKIM signature; it can have any value and the message will still pass DKIM validation.
As mentioned last time, including a header that doesn't exist in the DKIM signature signs its absence; if that header is then added to the message, the DKIM signature will become invalid. DKIM signing things that aren't there is sometimes called oversigning a header; you're not just signing what's present, you're also signing what's not. As a corollary of this, if you want to seal a message against having extra copies of some headers added, you can deliberately oversign existing headers. This is done by including their names an extra time in the h= list; the first time signs the existing header, and the second time signs that there's no second header. So if we wanted to make sure no one added a second 'From:' to a message, we'd sign 'h=from:from:[....]'.
One reason to oversign existing headers that should only appear once is that anyone who adds a second 'From:', 'Date:' or whatever to your message is probably up to no good. Another reason is that it's hard to predict which instance of the header a mail client will show to people reading the message, and there are probably some mail clients that will show the wrong instance of the header (the instance that isn't covered by your DKIM signature and so can be set to anything by an attacker).
This creates several options and decisions:
- do you make it so that certain headers can't be added to the message later, like the List-* and Resent-* families, or allow them to be added later?
- what headers do you sign if they're present? For example, should you sign Resent-* or List-* headers at all?
- do you oversign some existing headers so that no additional copies can be added?
Based on a quick skim of email that I have handy, relatively few sources of mail seem to be oversigning existing headers. However, GMail does oversign at least some email for core headers like From: and Subject:. Since Google is one of the eight hundred pound gorillas of email, if they're doing it people's DKIM signature validation is at least prepared to cope with this.
(I suspect that having two From:, Subject:, or so on headers trips enough spam detection systems that attackers don't normally do it.)
The issue of what headers to include in your DKIM signatures
Increasingly, you have to sign your outgoing email messages with DKIM. When you use DKIM to sign things, in one sense you're signing an abstract 'email message', and in another, more concrete sense, you're signing the email body plus some of the email message headers. You might innocently think that the message headers to sign are standardized and obvious, but I've recently learned that neither is the case due to a recent discussion on the Exim mailing list. Different mail systems may sign different sets of headers in ways that are more or less aggressive, and some of these ways have downstream effects.
(This is especially relevant to Exim, where the default configuration of what headers to sign is perhaps somewhat aggressive.)
A basic part of DKIM signing is that if a message doesn't have a particular header and you include it in the DKIM signature headers anyway, what you're doing is signing that there is no such header in the email; basically, the header is interpreted as having a null value. If someone adds the header later, it will have a non-null value and so fail the DKIM signature check. Signing nonexistent headers is important if you think that adding them would change the meaning of the message as people perceive it (or as they see it).
As far as what headers to include goes, RFC 6376 provides relatively little guidance in section 5.4 and then a big and somewhat questionable list in section 5.4.1. Some headers are in practice part of the meaning of the message as people reading it will perceive things; in this category I'd include From: (which is required anyway), Subject: and Date:, and probably To:, cc:, and Reply-To:, and in practice I'd roll in In-Reply-To and References and some others. Some headers will change the interpretation of the message body if modified so must be protected by the DKIM signature; this includes all MIME related headers.
But then you have headers that may or may not change what you see as the meaning of the message if they're added to it after your signature. In this category are both the Resent-* family of headers for resent messages and especially the List-* family of mailing list headers. In some environments, whether a message was sent directly to people or came through a (visible) mailing list matters, as does what mailing list; in those environments you probably want to include the List-* headers in your DKIM signatures. But in other environments, this is not critical and in fact your people may be sending messages to outside mailing lists and want this to not break the DKIM signatures of their messages so the post-mailing-list version of their email is still accepted by, for example, GMail.
(You can have a similar discussion about Resent-*. Maybe these headers should never be signed, maybe they should be signed only if they're present, and maybe they should always be signed so that if someone visibly resends a signed message, it no longer passes DKIM verification.)
Now that I'm aware of this issue, we're probably going to change away from the Exim default (which signs all of the section 5.4.1 headers, plus the MIME headers) to something where we definitely don't sign the List-* headers and probably don't sign the Resent-* headers.
PS: One of the reasons to not sign Resent-* and List-* headers is that in both cases, you can do resending and mailing lists without changing the headers at all. Breaking DKIM signatures if people actually do add headers thus only encourages them to not add the headers; since adding the headers is useful and nice, we shouldn't discourage people from doing so.
Having ClamAV reject email using the Malwarepatrol database seems unwise
In practice, ClamAV is both a virus and malware recognition engine and a collection of malware signatures. ClamAV only comes with a limited set of signatures, so supplementing it with additional third party sources is popular (and perhaps almost essential). Often people use update tools and scripts to configure and fetch these additional signatures, such as Fangfrisch. One of the popular providers of third party signatures is Malware Patrol, who have a number of tiers of access, including a (free) tier for educational institutions. Since we are an educational institution, we signed up for this tier and added it to the configuration of the third party update script we were using at the time so that it would be part of our email anti-spam filtering (when we switched over to ClamAV from our prior solution). Well, we thought we'd added it; in fact we'd made a configuration mistake such that we were silently failing to fetch the Malware Patrol database. We only noticed and fixed this mistake when we switched to Fangfrisch for our third party updates.
Soon afterward, our logs started reporting rather a lot of Malware Patrol hits and some people here started complaining that email to them was being rejected. Investigation showed that the rejections were from Malware Patrol signatures and the ones we could decode had what I would call alarmingly broad text matches that they were looking for (Malware Patrol uses ClamAV's body-based signature content format, generally with just a string it's looking for).
(One reason we couldn't decode what some Malware Patrol signatures were matching was that the Malware Patrol data is updated frequently, with signatures regularly being removed.)
Malware Patrol is fairly open and unapologetic about these broad matches in an article called Whitelisting for Block Lists. They specifically say:
Malware Patrol’s #1 goal is to protect customers from malware and ransomware infections. These days, this can mean blocking mainstream domains. Consequently, our customers report potential false positives for sites like docs(.)google(.)com, drive(.)google(.)com, dropbox(.)com and github(.)com. Systems like Google Docs serve files from their root directories. This forces some block list formats to then block the entire domain, frustrating users.
Although Malware Patrol doesn't say this explicitly, it appears that the ClamAV database format is one such format that sometimes forces them to block entire domains like 'drive.google.com' (we observed this in one signature). They suggest filtering their database before using it, but this has a number of problems; the ClamAV format is hex-encodes the ASCII bytes, for example, and on a larger scale it would mean we'd only be excluding things after people here had run into problems and reported them to us.
I don't fault Malware Patrol for their choice. The balance between false positives and false negatives is not one with a clear single answer, and Malware Patrol seems to have come down on the side of not having false negatives, even at the cost of false positives. But it does mean that Malware Patrol's objectives and ours aren't in alignment, as we care more about avoiding (too many) false positives than we do about avoiding every last false negative.
Our resolution to this was to take Malware Patrol out of our third party ClamAV data sources. I'm sure there are situations where using their database as part of ClamAV screening makes sense, but my view is that if you're rejecting email based on ClamAV signature matches, you likely can't use Malware Patrol's data. It's too dangerous unless you have a quite high tolerance for false positives. Even in a system where a Malware Patrol signature match only contributed to a message's spam score, I think you could only really add a modest increase in the odds of the message being spam.
(As far as I know, ClamAV stops looking once it's found a signature and the order it checks signature databases isn't documented. This means there's no way to tell it to check signature databases you trust more before Malware Patrol.)
PS: I don't know how common it is to use ClamAV signature matches to reject email, but it is, for example, an obvious way to configure Exim, especially since Exim's malware scanning documentation does this in its example.
Email anti-spam (and really all anti-spam) is all heuristics now
On the Fediverse, I noted something:
Back in the days, one of the things some people said about DNS blocklists in general and sometimes Spamhaus in particular was that they were opaque, capricious, and didn't actually validate what they were putting in their blocklists, so who knows what could wind up in there for who knows what reason. Those people would take this incident as a validation of their view.
(I was going to say that this was a long standing IP address used to send Ubuntu security announcements, but it looks like we only just started to get them from this IP, although the entire IP range is owned by Canonical.)
I have bad news for such people. This is what all email anti-spam systems are doing today. There are no effective anti-spam systems that are based only on sure positive signs of spam. Everything is an opaque black box full of heuristics and uncertainty, with hopefully occasional misfires that are hopefully not too spectacular. Sometimes people hand write rules and try to assess them, sometimes people take straightforward statistical approaches (eg, Bayesian scoring), and sometimes companies go for the complicated statistics that are generally known as 'Machine Learning' or these days 'AI' (in press releases, at least).
This is not an accident and it's not because people are lazy. It's because anti-spam isn't working against a blind natural phenomenon; instead, anti-spam is engaged in an iterated game against human driven spam. If there's a sure-fire signal of spam that can be used to reject or filter email, the humans driving spam are highly incentivized to get rid of it, and only the ones who are successful at that will survive.
This is simply one of the prices that spam exacts from us. We can no longer live in a world of certainty, where we can be confident that our anti-spam systems are right about things. And sometimes we'll see things that are so obvious (to us humans, on the spot, only having to look at this one incident) that they make us have sad faces.
(There's also the related issue that no one can afford to pay enough humans enough to constantly be evaluating and updating anti-spam rules and heuristics all of the time. All effective anti-spam systems have to operate partially automatically, and sometimes that will pass things that an alert human would not have.)
You should delete the 'User-Agent' header from outgoing email
We all know about the HTTP User-Agent header, which browsers and other web things send to web servers. The nominal purpose of this is covered in RFC 9110 section 10.1.5, and it's not terrible, but in practice websites have abused the header for years (if not decades) and the whole thing is a major mess (eg). A very long time ago, some mail clients decided that they'd advertise by adding an 'X-Mailer' header to email they sent, with their name in it. Somewhat more recently, various mail clients decided that they would do this using a 'User-Agent' header (sometimes in addition to an X-Mailer header); one common example is Thunderbird.
I have come to think that this is a bad idea and that you should configure your mail submission server to strip User-Agent (and probably also X-Mailer). First off, leaving this header in leaks information about your users to various people. With the way that the Internet has evolved, hiding this information is now the right answer, much like hiding user IPs turned out to be the right call. If you need to know client and device usage information for your own purposes, log the header value before you delete it (but understand that not all clients may add it in the first place).
(This information leaks not just to the people who your users send email to, but also to the people who operate the receiving email servers. These days that often means Google and Microsoft.)
Second, with the way that the spam filtering landscape has evolved into an unpredictable mess based in large part on opaque signals, other people's mail servers may well decide that they don't like certain User-Agent values. If your people are using one of those mail clients (possibly authentically, unlike spam that forges such a User-Agent), their email will be less likely to get through. Since not everything provides a User-Agent field in the first place, I believe that stripping it out entirely is not likely to be harmful, especially by comparison.
(You might feel that using User-Agent in this way is morally wrong, but other mail servers don't care about your feelings and anyway they may not be explicitly looking at 'User-Agent' as such. They may well be just feeding everything in as barely classified text and letting some pile of math look for correlations, so any header and any header value or part of its value that has correlations will be used.)
In my view, giving other people's large and opaque mail systems fewer reasons to consider real email from your people to be spam is a good reason all by itself. The privacy benefits just tilt the situation even more toward removing any User-Agent header that mail clients may have added.
(As a corollary, it's long since past time that mail clients stopped adding this header. No one is paying attention to it and it's a little leak of private information.)
The types of TLS seen on our external MX (as of April 2023)
On the Fediverse, I said:
Today's sysadmin tip: if you don't want to be depressed, don't look at how many other mail servers are still connecting to your external mail gateway with TLS 1.0, and especially not exactly who they are.
Today I feel like providing some statistics on that, partly for my own interest. All of these are over the past full nine days, which means that they mostly cover the end of April 2023 (plus May 1st).
Over this time we accepted 94,037 messages, of which 62,885 were encrypted with some version of TLS. The TLS versions used break down like this:
36426 X=TLS1.2 26209 X=TLS1.3 229 X=TLS1.0 21 X=TLS1.1
After my Fediverse post, I'm actually surprised to see such a low usage of TLS 1.0 and 1.1. I'm pleased to see that TLS 1.3 is so close to TLS 1.2.
(I think what I was seeing in my Fediverse post was that outside mailers were making a handful of connections a day with TLS 1.0 and TLS 1.1. At the time the TLS 1.0 connections stood out more.)
I don't particularly know why TLS 1.1 is so uncommon compared to TLS 1.0. It may be that TLS 1.1 was only the latest version of TLS for a few years (based on Wikipedia's dates). There was probably a relatively narrow window of time for people to have developed and shipped TLS 1.1 products (and then never updated them to TLS 1.2).
Ubuntu 22.04's version of Exim conveniently formats the full cipher name in a way that makes it easy to get a top level view of the broad signature schemes in use:
25774 X=TLS1.3:ECDHE_X25519 19678 X=TLS1.2:ECDHE_SECP256R1 11159 X=TLS1.2:ECDHE_SECP384R1 2916 X=TLS1.2:ECDHE_SECP521R1 2599 X=TLS1.2:ECDHE_X25519 435 X=TLS1.3:ECDHE_SECP256R1 203 X=TLS1.0:ECDHE_SECP256R1 74 X=TLS1.2:RSA 26 X=TLS1.0:RSA 16 X=TLS1.1:ECDHE_SECP521R1 5 X=TLS1.1:RSA
Overall, there were 34 different full cipher suites used, and so I'll give a little breakdown by TLS protocols (partial for TLS 1.2):
13796 X=TLS1.3: ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_128_GCM: 128 11960 X=TLS1.3: ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM: 256 424 X=TLS1.3: ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM: 256 18 X=TLS1.3: ECDHE_X25519__RSA_PSS_RSAE_SHA512__AES_256_GCM: 256 11 X=TLS1.3: ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_128_GCM: 128 13377 X=TLS1.2: ECDHE_SECP256R1__RSA_SHA512__AES_256_GCM: 256 11089 X=TLS1.2: ECDHE_SECP384R1__RSA_SHA256__AES_256_GCM: 256 3719 X=TLS1.2: ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_128_CBC__SHA1: 128 2880 X=TLS1.2: ECDHE_SECP521R1__RSA_SHA512__AES_256_GCM: 256 2037 X=TLS1.2: ECDHE_SECP256R1__RSA_SHA256__AES_128_GCM: 128 1820 X=TLS1.2: ECDHE_X25519__RSA_SHA512__AES_256_GCM: 256 497 X=TLS1.2: ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_128_GCM: 128 433 X=TLS1.2: ECDHE_SECP256R1__RSA_SHA512__AES_128_GCM: 128 [...] 16 X=TLS1.1: ECDHE_SECP521R1__RSA_SHA1__AES_256_CBC__SHA1: 256 5 X=TLS1.1: RSA__AES_256_CBC__SHA1: 256 203 X=TLS1.0: ECDHE_SECP256R1__RSA_SHA1__AES_256_CBC__SHA1: 256 26 X=TLS1.0: RSA__AES_256_CBC__SHA1: 256
(I've added spaces after the :s for better line wrapping.)
As we can see here, TLS 1.2 contributed the largest diversity; it has 25 different full cipher strings. I believe this reflects a wide diversity of opinions in the sending MTAs, because the Exim documentation says that the client (here, the sending MTA) picks the preferred cipher if you're using GnuTLS, as the Ubuntu Exim is.
Sidebar: the TLS 1.2 RSA ciphers
44 X=TLS1.2: RSA__AES_256_CBC__SHA1: 256 18 X=TLS1.2: RSA__AES_256_GCM: 256 12 X=TLS1.2: RSA__AES_128_CBC__SHA1: 128
I don't know how horrified I should be here.
The chain of landing web pages that I saw for a phish spam today
Over on the Fediverse, I shared a phish-related discovery:
Today's discovery: people hosting phish landing forms in IPFS and using Cloudflare's IPFS gateway to do the work of web access to them. Nicely played. Everyone is going to point fingers at everyone else.
(As usual the email has a different URL, with a 'this is our secure document link' that takes you to the IPFS hosted form.)
Let's be a little bit more specific, because it's a useful example of just how complicated these things can be.
The email was a 'X has shared a file with you' email with a link to a page on what claimed to be a travel company's Sharepoint aka OneNote site (under '<...>-my.sharepoint.com', which you probably aren't going to be able to block). Based on the URL, this may have been a page created by and for a particular user, instead of a corporate page, meaning that just this user had their Sharepoint access compromised. This page said:
<company> transmitted a secured RFQ itinerary Doc
To view the doc, click the link below
(The spelling here is authentic.)
That link took you to a cloudflare-ipfs.com URL, which displayed an official looking Adobe 'Verify Your Identity' thing asking you to sign in:
You've received a secure file
[PDF icon] 58.3 Kb
To receive and download this PDF file , please enter specific professional email credentials that this document was sent to.
(On the one hand, this is your web browser clearly asking you to authenticate to see a PDF. On the other hand, Adobe has clouded up all its PDF programs, so sure, why not assume that Adobe runs a secure PDF sharing thing for people as part of that.)
(This uses the same idea of 'you must authenticate to view this PDF' but with more steps than the other recent-ish time I've seen this trick. That this bounces you through a website may make it more plausible to people, since having to authenticate to access things for you on the web is not uncommon. It may raise large red flags to technical people who stop to think about the mechanics, but we're not really the target audience for this attack.)
The case of the very wrong email
Over on the Fediverse, I shared a discovery from our mail logs:
In the 'that's not how you do it' category, spotted in our email reject logs today:
(We rejected it for having an absurdly long line, over 200,000 bytes, which appears to have been almost all of the message.)
The MIME Content-Transfer-Encoding header is supposed to tell you the encoding of the MIME part in question, including the implicit top level part of the email. Typical values are things like '7bit', '8bit', 'quoted-printable', or 'base64'. Needless to say, this email's C-T-E is complete garbage, and a picky email client would say that it couldn't decode the message because it doesn't understand the 'amazonses.com' encoding.
(I suspect that real clients treat this as an unset C-T-E and either assume they have text or try to guess among the options.)
All of this email appears to be spam, of course. And the message has other anomalies besides the absurdly long lines in the body. They seem to have a consistent envelope sender domain (that I'm not going to mention for reasons), but the headers follow an unusual pattern. If the email is sent to 'USER@<our-domain>', all of the samples I've checked have the following header setup:
From: [...] <support@USER.net> Sender:USER@<our-domain> Message-ID: <...email@example.com..org> Content-Type: text/html Content-Transfer-Encoding: amazonses.com
That 'Sender:' header is malformed, and obviously you aren't supposed to have a constant Message-ID. Obviously some of the addresses are made up (and forged, for the sender address); many of the From: domains probably don't even exist. While the envelope sender domain stays constant, the local addresses do vary. The current sending IP has also been consistent over today.
(At the moment the MX for the envelope sender domain is outlook.com, and they reject a random claimed envelope sender address. Of course this spammer could be forging the domain of an innocent bystander, which is why I've decided not to mention it.)
The obvious speculation about where the gigantic line comes from is that the messages have extremely bloated HTML of some sort and it's all been crammed on to one line. I don't know what you do to get a 200,000 to 340,000 characters of HTML in an email message; maybe they're including images as inlined 'data' URLs.
How to block people's automatic mail forwarding (to GMail, at least)
Suppose, hypothetically, that you're the kind of person who is certain that your email is so sensitive that it should never be automatically forwarded. If you send email to firstname.lastname@example.org and the person likes to forward their email to GMail, well, tough. Your email is too important; they can read it through example.org or not at all. Given the anarchy of Internet email, it sounds like this would be hard to achieve, but don't worry; modern email standards have your back here, at least for places (like GMail) that generally respect them.
Here's what you do. First, configure a strict DMARC policy for your domain, one that tells receivers that you want them to reject any email that doesn't pass DMARC. Then, set up a restrictive SPF policy, one that definitely only passes things sent from your server. Finally, the important step: don't sign your outgoing email with DKIM.
Since you have a strict DMARC policy, receivers like GMail will
reject email with a '
From:' header with your domain that doesn't
pass DMARC checks (this is DMARC alignment). Since you do have
a (restrictive) SPF record, email send directly from your email
servers will pass SPF checks and so pass DMARC alignment. But since
you don't DKIM sign messages, if GMail receives email from anywhere
else with your domain in the From: header, the email will fail
DMARC; it can't pass a DKIM check because there's no signature, and
it can't pass a SPF check because it doesn't come from you.
Some automatic forwarding will change the envelope sender (the SMTP MAIL FROM) so that it will pass other people's SPF checks (this can be done with SRS or other mechanisms). But very little automatic mail forwarding changes the From: header address, partly because doing so makes it much harder for the person receiving it to do things with the email. And if the forwarding system adds its own DKIM signature, nothing really changes because the signature won't be for your domain and won't count for DMARC alignment.
I regret to inform you that there are mail systems out in the world who are actually doing this, although perhaps they aren't doing it deliberately. Maybe their DKIM signing has broken, or doesn't cover all of the email they sent, or just never got implemented. These people even send mail to people at universities, I assume deliberately. Not all of that email gets through.
(People can of course still manually forward your messages, because manual forwarding generally creates a From: header with their email address, and now what matters is their DMARC policies, DKIM signatures, and SPF records, which their email probably passes if they want it delivered.)
PS: Possibly Google's SMTP rejection messages that I've seen for this have been incomplete, in that maybe Google wouldn't have been as insistent on DMARC alignment in other situations. I saw this with Message-ID headers.
(SMTP email long ago stopped being a fully predictable or understandable system, as systems took increasing measures to defend themselves against spam.)
Our current plague of revolving .top and .click spam email domains
Email spam is somewhat like the weather, and much like the weather I don't talk about it much any more. However, every so often something unusually unpleasant happens (in both of them). Our current irritation in spam weather is what I suspect is one particular spammer that operates using a rapidly changing flux of spam domains in .top, .click, and on some days .us, using a distinctive (but not really machine matchable) pattern of tagged envelope senders.
The typical pattern of envelope senders are ones that look like this:
This '<phrase>-<user>=<domain>@<random>.(top click us)' envelope sender is quite human recognizable and is clearly tagged, but it's not all that easily matched without false positives. The tagged envelope sender is less useful than it looks, because none of these domains actually accept email.
The spammer is fast moving at changing both sending domains and sending IPs. Their domains and IPs tend to wind up listed by people like Spamhaus within an hour or three, but by then they've moved on. They don't seem to reuse domain names very much (or very fast, when they do reuse them), but in a spot check they did reuse IP addresses over the past couple of days, perhaps as they fall out of the SBLCSS and similar DNS blocklists. Possibly the spammer reuses domain names less often due to them expiring from DNS blocklists more slowly than IPs.
(In fact now that I'm looking at this seriously, this spammer appears to be only using three /24s for the past week or so.)
Unfortunately our current anti-spam software (rspamd) doesn't immediately recognize this mail as spam (although once things are DNSBL listed it can do better). GMail is wise to their tricks, of course, and so email from this spammer to people here who forward their email to GMail is rejected by GMail at SMTP time, giving us bounces that pile up in our queues as we try to deliver them to the spammer (who is, as mentioned, not taking email). The messages have valid DKIM signatures and even pass SPF checks (to my amusement the spammer thoughtfully lists the domain's sending IP in their SPF record).
(GMail typically rejects the email with messages about the reputation of the sending domain being too low, but as I've seen with Message-IDs, GMail's rejection messages aren't necessarily anywhere near the whole truth.)
In a spot check of these domains, DNS service is being provided by Cloudflare, perhaps via some free plan or perhaps as part of the registrar's offering. WHOIS for recent domains lists the registrar as NameSilo LLC (although the domains might have been obtained through a reseller), who appear to offer very low up front costs for registering domains in some of these TLDs. Still, the churn in domain names suggests to me that the spammer probably isn't paying for them in one way or another.
The side effects of this particular spammer are sufficiently annoying that I may take some specific steps to deal with them. While there are a bunch of clever, complicated options, it's possible that quite brute force ones would be sufficient.
(Mostly I'm irritated that people are letting them get away with going through so many domains. Domain registration is supposed to cost money, and domains aren't supposed to be expendable things for spammers. Yet here we are, with 'fast flux' domain names.)