2026-09-10 · 6 min read

The CrowdStrike outage was a bounds check against the wrong array, and it would pass most code reviews

Two summers ago, 8.5 million Windows machines blue-screened within about an hour of each other, airports switched to handwritten boarding passes, and Delta cancelled around 7,000 flights over the following five days. Everyone remembers the images. What almost nobody remembers is the actual defect, which is a shame, because when CrowdStrike published its root cause analysis the mechanism turned out to be one of the most ordinary bugs in software: an index was bounded against the wrong length. That's it. The kind of thing you'd nod past in a diff at 4pm.

What the RCA actually says

CrowdStrike's Falcon sensor has a component called the Content Interpreter that evaluates "Rapid Response Content", essentially rule files pushed to every machine several times a day, against a fixed set of inputs the sensor collects. In February 2024 they shipped a new template type for inter-process communication events. The RCA puts the defect in one sentence: "The new IPC Template Type defined 21 input parameter fields, but the integration code that invoked the Content Interpreter with Channel File 291's Template Instances supplied only 20 input values to match against."

So the validator on the build side believed there were 21 fields. The interpreter on the sensor side was handed 20. That mismatch sat in production for five months without doing anything, because every rule shipped in that window used a wildcard for the 21st field, and a wildcard never gets read. Then on July 19 at 04:09 UTC a rule went out with a real value in field 21, the interpreter tried to read one past the end of a 20-element array, and in kernel context an out-of-bounds read is a crash, not an exception. The update was reverted at 05:27 UTC. Seventy-eight minutes was enough to reach 8.5 million devices, by Microsoft's count, and every one of them needed a human with a keyboard to recover.

The RCA is unusually candid about why testing missed it: the stress tests used a template with a wildcard in the 21st slot, so the one test that would have caught the mismatch was structurally incapable of catching it. Which means the last gate with any chance was the same one it always is. Someone reviewed the validator, and the validator checked a bound. Just not the right one.

The same bug, as a PR you'd actually see

We rebuilt the mechanism as a small Node PR in the library, and I want to show you the diff because the history is only half the point. The other half is how little the bug looks like a bug. The requirements say: reject any rule whose field index is outside [0, INPUT_REGISTER_COUNT), so that the hot path can index the input array without a per-read check.

Read line 14 the way you'd read it in a real review. There's an integer check. There's a lower bound. There's an upper bound. There's an error message with the offending value in it. This is what a careful validator looks like, and if the PR description says "validated against the sample channels" (which it does, and which was true), your brain files it under "defensive code, good" and moves on to the interesting part.

Why this is a review bug, not a testing bug

It's tempting to file CrowdStrike under "they should have had better tests," and the RCA does list a missing test as one of its findings. But think about what that test would have needed to be: a channel file with more rules than input registers, containing a rule that references a register index above 19. Nobody writes that test unless they've already noticed that the two lengths are different things, and the moment you've noticed that, you've found the bug without the test. The test is downstream of the insight, and the insight is a review skill.

Specifically, it's this question, which I'd put in the top five most valuable questions a reviewer can ask about any diff: when you bound an index, are you bounding against the array you are about to read? Not against something that looks like a length. Not against a count that's usually the same. Against the actual array, at the actual read site, and if the read site is in a different function (which it always is, that's the whole point of validating up front) then you go and look at it. In the diff above that's a ten-second hop from line 14 to line 23. In CrowdStrike's codebase it was a hop from the Content Validator to the Content Interpreter, two different components owned by different people, and nobody made it.

That pattern, validation living far from the read it protects, is everywhere in modern code and it's especially common in AI-generated PRs, which tend to produce beautifully structured validation functions whose bounds are plausible rather than derived. The model doesn't know which length is the right one any more than a tired human does. It knows that a bounds check goes here and that a length variable goes in it, and it picks one that's in scope.

What it cost, in case the argument needs numbers

Delta's own quarterly filing puts the direct revenue hit at $380 million, plus $170 million in non-fuel expense, for one airline. The lawsuit that followed is still working its way through a Georgia court as of the last public order I could find. Nobody has published a global figure that isn't a guess, but the one that's hard to argue with is that a comparison against the wrong variable, on one line, in one validator, cost more than most companies will ever be worth.

You won't review the Falcon sensor. But you will review a validator this month, probably this week, and it will check a bound, and the bound will look right. The only question is whether you've trained the reflex to go and find the array.