2026-09-09 · 6 min read

Knight Capital lost $460 million in 45 minutes because a PR reused a retired feature flag

Knight Capital was one of the biggest market makers in US equities. On 1 August 2012 it took 45 minutes to lose more than the company was worth. No hack, no rogue trader, no exotic instrument. A deploy that reached seven of eight servers, and a feature flag that somebody reused because it was "already plumbed through every environment." If you've ever approved a PR that gated a new feature behind an existing config key to save a schema change, this is the story of what that looks like at scale, and the SEC wrote it up in more detail than most postmortems ever get.

What the SEC order says happened

Knight's order router, a system called SMARS, still contained code for a feature called Power Peg that the firm had stopped using in 2003. Dead, but present and callable. In 2012 they built a new routing feature for the NYSE's Retail Liquidity Program, and instead of adding a new switch for it, the order says plainly: "The new RLP code also repurposed a flag that was formerly used to activate the Power Peg code. Knight intended to delete the Power Peg code so that when this flag was set to 'yes,' the new RLP functionality, rather than Power Peg, would be engaged."

Intended to. The deploy went out in stages over a week, and "one of Knight's technicians did not copy the new code to one of the eight SMARS computer servers. Knight did not have a second technician review this deployment." So on the morning of 1 August, when orders started arriving with the repurposed flag set to yes, seven servers ran the new feature and the eighth ran Power Peg, a routine nobody had touched in nine years, whose fill-counting had been moved elsewhere in 2005 without anyone retesting whether Power Peg still worked if called. It didn't. It sent child orders for each parent "in rapid sequence ... without regard to the number of share executions Knight had already received."

The numbers from the order: 212 incoming orders, millions of child orders, over 4 million executions in 154 stocks, 397 million shares, a $3.5 billion long position and a $3.15 billion short position accumulated in 45 minutes, and a realized loss of $460 million. And then the detail that should be taught in every ops course: when they tried to roll back, they uninstalled the new code from the seven good servers, which meant all eight were now running Power Peg. The rollback made it worse. Knight was acquired the following summer and the name is gone.

The bug is a naming decision

Strip out the trading and it's this: one flag name, two meanings, and a codebase where the old meaning still had reachable code behind it. Every environment had the flag set to off, which is why the reuse looked free. It isn't free. A dormant flag is only dormant while nobody flips it, and the entire point of the PR was to flip it.

We rebuilt the mechanism as a Node PR in the library, and I'd argue it's harder to catch than the original, because the author explains their reasoning and the reasoning sounds sensible. Priority tagging for carrier submissions, gated behind the existing priorityDispatch key because "it's already plumbed through every environment and has sat dormant since the old path was retired, so ops can enable the feature without a config-schema change." The requirements even say not to delete the legacy block in this PR, its removal is tracked separately. So the legacy code stays, which is fine, that's normal. Here's the diff.

Add priority tagging to carrier submissions · src/fulfillment/dispatch.js -1 +12

17 // `priorityDispatch` flag has been off in every environment since retirement;

18 // this block stays in place until the dead-code cleanup lands (FUL-482).

20 function legacyPriorityDispatch(order, carrier) {

21 carrier.submit({ orderId: order.id, sku: order.sku, qty: order.qty });

22 }

24+function buildSubmission(order, flags) {

25+ const submission = { orderId: order.id, sku: order.sku, qty: order.qty };

26+ if (flagOn(flags, 'priorityDispatch')) {

27+ submission.priority =

28+ order.valueCents >= EXPRESS_THRESHOLD_CENTS ? 'express' : 'standard';

29+ }

30+ return submission;

31+}

41 function runTick(orders, carrier, state, flags) {

42 for (const order of orders) {

43 if (flagOn(flags, 'priorityDispatch')) {

44 legacyPriorityDispatch(order, carrier);

45 continue;

46 }

The new code is lines 24 to 31 and it is correct. Threshold logic, right. Byte-for-byte identical output with the flag off, right. Tests on the default config, green. The bug is not in a single line the PR added. It's in the relationship between line 26 and line 43, which the PR didn't touch, and which most reviewers won't scroll to because it's outside the diff.

What a trained eye flags here

runTick checks the same key first. With the flag on, every order takes the legacy branch on line 44 and the continue on line 45 skips the ledger-based path entirely, so buildSubmission never even runs. Nobody gets a priority field. Instead, legacyPriorityDispatch, which has no acked or submitted check, resubmits every order in the list to the live carrier on every tick, including the ones already shipped, forever. That's Knight's eighth server, in twenty lines. And exactly like Knight, the code behaves perfectly until the day someone flips the switch, so no test on the current config can see it.

The review comment: "line 26 reuses priorityDispatch, but line 43 still checks that key and routes to the legacy path with continue, so enabling the feature revives the retired dispatcher (no ledger, resubmits acked orders every tick) and never reaches buildSubmission. Needs its own key, e.g. priorityTagging, and priorityDispatch stays off until FUL-482 deletes the block." One flag, one meaning, until the old meaning has no code left to run.

Review it yourself before the reveal, if you'd rather. The library keeps the incident a secret until after your verdict.

The review question that catches this class

It's one grep. When a PR gates anything behind a config key that already exists, search the codebase for every other read of that key and ask whether each one still means what the PR thinks it means. Thirty seconds. In this diff the other read is fifteen lines up. In Knight's codebase it was a routine last used when the iPod was new. The distance changes; the question doesn't.

What makes it a genuinely hard review skill, rather than a checklist item, is that the reuse is always justified in the PR description, and the justification is always true. The flag is already plumbed. It has been off everywhere. Adding a new key is a schema change. All of that is real, and none of it is the point. The point is whether any reachable code still answers to the old name, and the author, having just written the new code, is the person least likely to go looking. That's what you're for.

One more thing from the order

The sentence I keep coming back to isn't about the flag. It's "Knight did not have a second technician review this deployment." Seven servers got the new code, one didn't, and the reason nobody noticed is that nobody was looking. The flag reuse turned a bad deploy into a catastrophe, but it was the missing second pair of eyes that let a bad deploy happen at all. Review is the cheapest control in the whole chain, and it was the one they skipped.

Try a review

Already plumbed through every environment. Off everywhere. What could go wrong.

Review the Knight-mechanism PR and 200+ others with planted bugs. Free, no signup. Canonical review after your verdict.

Review this PR →