Google recently wrote a (blog) article, On Fire Drills and Phishing Tests, which discusses the early history of what we now call fire drills. As the article covers, the early "fire evacuation tests" focused mostly on how individual people performed, complete with telling people that things were their own fault for not doing the evacuation well enough. It then analogizes this to the current way "phish tests" are done. As I read this, I had a reaction on the Fediverse to the general thought of fire drills and phish tests:

In re comparing fire drills to phishing tests[1], if phishing tests were like fire drills, they would test the response to a successful phish. Was the person phished able to rapidly report and mitigate things? Do the organization's phish alarms work and reach people? Etc etc.

Current "phishing tests" are like testing people to see if they accidentally start fires if they're handed (dangerously) flammable materials. That's not a fire drill.

1: <fediverse link>

The purpose of fire drills is to test what happens once the fire alarm goes off and to make sure that it works. Do all of the fire alarms actually generate enough noise that people can hear? Are there visual indicators for people with bad or no hearing? Can people see (or hear) where they should go to get out of the building? And so on and so forth. In other words, fire drills test the response to the problem, not whether the problem happens in the first place.

(They also somewhat implicitly test if people respond to fire alarms, because if people don't you have another problem.)

As I mentioned in my Fediverse post, current "phish tests" aren't doing anything like this. Current "phish tests" are testing people to see if they recognize and (don't) respond to phish messages (and then blaming people if they don't handle the phish right, which is one of the things that the Google article is calling out). A "phish drill" that was like a "fire drill" would test all of the mitigation and response processes that you wanted to happen after someone fell for a phish, whatever these were. Of course, one awkward aspect of testing these processes is that you actually have to have them and they need to be made effective. But this is exactly why you should test them, just as part of the reason for fire drills is to make sure you have enough alarms, evacuation routes, and so on (and that they all work).

(I personally think that current blame the person "phish tests" are counterproductive in an additional way not covered by the Google article, but that's another entry.)