2021-12-24
Sadly, my experience is that big commercial anti-malware detection is better
For reasons beyond the scope of this entry, for the past couple of years I've been running a large commercial anti-spam system (and its malware recognition) side by side with what we could put together with ClamAV and some low-cost commercial ClamAV signature sources. More or less from the beginning it's been clear to me that our commercial system was recognizing malware that ClamAV was not. Some of this was new things that we could add to our manual recognition and rejection, but at this point another significant source of missed ClamAV recognition is (still) malware in Microsoft Office files.
This is not really a result that I was hoping for. Our commercial anti-spam system has been on vendor life support for more than a year, so its recognition engine definitely isn't being updated for new capabilities and who knows how much its signature database is being updated. Despite that, it's still ahead of a well regarded open source malware detection system.
Some amount of bad email makes it through both ClamAV and our commercial anti-spam system and is then forwarded on to elsewhere by some of our users. These days, that elsewhere includes both Office365 and GMail. Trawling our logs suggests that both of these recognize and reject even more malware than we do, although this effect is somewhat entangled in them also recognizing more spam than we do.
This is not really surprising. Large providers of email and of anti-spam services have more resources for both improving their scanning engines and coming up with signatures and danger signs. They see more email (one way or another) and can build more sophisticated systems to analyze it in various ways. Greater volume with automated analysis and feedback systems can mean faster responses to new malware. It's not really surprising that the open source and small commercial firms can't match this.
(One suggestive thing is that our commercial anti-spam software provider is not getting out of the anti-spam business. Instead, it's moving to having only a cloud filtering option, where you run your incoming email through their cloud systems. This gives them far more aggregate visibility into potential malware and makes responding to it much faster. I suspect that they were pushed to this partly to match the malware filtering quality of the big providers like Google and Microsoft.)
PS: For Microsoft Office files specifically, it might be possible for us to build something using oletools, and we may have to try to, just to not let too much bad stuff through once we can no longer use the commercial anti-spam software.
(This is one unhappy aspect of how running your own email is increasingly an artisanal choice. It's possible that a lot of manual tuning and adjustment and software will get us to something close to the quality of big commercial providers, but it's unlikely to be easy.)