An (accessible) explanation of the Flame malware's Windows Update compromise
June 8, 2012
I've been quite curious about the details of how the Flame malware managed to compromise Windows Update, as this sort of crypto wonkery is one of my interests. Unfortunately pretty much all of the coverage that I could find has either been vague and non-technical or written by specialists for other specialists. Since I've recently done a bunch of reading on this in an attempt to force things into my head, I'm going follow my usual tradition and write up what I think I understand and have guessed.
(The obligatory disclaimer is that I am not a security specialist so I may be misunderstanding parts of this.)
The story starts with Microsoft's Terminal Services licensing mechanisms. Part of enterprise TS licensing involves Microsoft signing a certificate for you which you then use to prove to the TS management software that you're properly licensed (and then it's apparently used to issue sub-certificates for client side licenses, which are tied to your organization through your certificate). This is a perfectly rational design with a number of useful features but Microsoft made at least three mistakes in the implementation.
The first mistake is that Microsoft was signing enterprise TS license certificates with a certificate chain that ran up to a general Microsoft Root certificate, one that is used as the certificate root for all sorts of things. This creates a situation where these enterprise TS certs and any sub-certs that they signed would be seen as 'signed by Microsoft' by general certificate verification code (such as that used by, oh, Windows Update).
(This chain running to a Microsoft Root CA is unnecessary; the TS license verification could perfectly well have its own detached and independent root certificate, one used only for license verification. Microsoft is apparently in the process of changing TS licensing to do just this.)
The second mistake is that these signed enterprise TS certificates were (indirectly) authorized as code signing certificates instead of being restricted to just license verification. In combination with the first mistake this becomes a 'signed by Microsoft' code signing certificate, which is good enough to compromise Windows Update in everything before Windows Vista.
(The authorization is indirect because the enterprise TS cert has no specific usage restrictions; instead the usage restrictions come from two of the intermediate certificates in the chain. Both of them specifically allow code signing plus some other things.)
Why the basic compromise doesn't work from Vista onwards is that the TS certificate signing system adds a custom TS-related extension (called 'Hydra') to the signed certificate that's marked 'critical'. Critical in X.509 means that if your code doesn't understand the field you're supposed to reject the certificate even if it otherwise validates. The TS license code presumably understands this field, but apparently the general Vista-and-later crypto library does not and thus would have rejected the certificate. This leads to the third mistake.
The third mistake Microsoft made is that the TS cert signing process was still using MD5, which is now quite susceptible to serious collision attacks (although practical ones remain what mathematicians call 'non-trivial'). The Flame authors exploited the weaknesses in MD5 to create a version of their signed TS certificate (with the same MD5 hash) that transformed the critical (must-be-handled) Hydra extension (along with several other fields) into just the data payload of a disused non-critical field that would be ignored by the signature verification code in the Vista-and-later crypto library that Windows Update uses. This finally created a 'signed by Microsoft' code signing certificate that would be accepted by all versions of Windows Update.
One of several scary things about this compromise is this was not a weakness in Windows Update's cryptography. The Windows Update people seem to have done everything or almost everything right; instead, they got compromised by a series of certificate handling mistakes in a completely different area of Microsoft (one that seems to have been neglected if not outright ignored before this).
(You can argue that Windows Update is so crucial that it should have had its own independent root certificate instead of relying on a general use Microsoft Root CA that was also the root of other certificate chains used for other purposes.)
PS: there seem to have been a number of other, less important flaws in how enterprise TS certificates were set up. From my reading so far, most of the other issues weren't important for the compromise; they just suggest that TS licensing never received the cryptography attention it needed. The one exception is that apparently enterprise TS certificate signing used predictable timing and field contents for things like the serial number, which is believed to have made the MD5 collision attack easier.
Sidebar: many references
Here is a bunch of the links that I've been relying on:
There are probably a number of other accessible explanations of the whole thing that I just didn't find in my web searching before I got overwhelmed by all of the breathless news coverage from the usual suspects.
* * *
Atom feeds are available; see the bottom of most pages.