2026-07-29 · 5 min read

AI now writes the tests for its own code, and that changes what green means

There's a solid empirical study on this now: Yoshimoto and colleagues analyzed 2,232 test-related commits across ten TypeScript projects for MSR 2026 and found that AI agents author 16.4% of all commits that add tests, ranging from 1.9% in a large enterprise repo to literally 100% in smaller ones. The surprising part is how good the tests look: coverage gains comparable to human-written tests, sometimes better, and twice the assertion density (a median of two assertions per test against one for humans). By every number a dashboard can show, the machine is a diligent test author.

Why the good numbers are the uncomfortable part

Coverage and assertion counts share a blind spot: they measure whether code paths get exercised and whether something gets checked, not whether the right thing gets checked. A test that pins down the wrong behavior scores exactly as well as one that pins down the spec. And when the same model wrote both the code and its tests, that's not a hypothetical: the tests are the code's self-portrait, painted by the same hand, asserting that the code does what the code does.

The field data shows where that leads. In Checksum's survey of engineering leaders, 74% rolled back AI-generated code that failed in a way its unit tests didn't catch: the suite was green all the way to the incident. GitClear's repo-scale data adds the darker sibling, a 47% rise in error masking, code that swallows failures so nothing looks broken. Put those next to the doubled assertion density and you get the honest summary: tests got more numerous, more thorough-looking, and less trustworthy as a signal, all at once.

Reviewing tests is now reviewing, full stop

Most reviewers treat the test file as the boring half of the diff, the part you scroll through nodding. That habit comes from an era when a human colleague wrote the tests as a check on their own understanding. The generated test file deserves the opposite treatment, and a few questions do most of the work:

• Do the assertions come from the requirements or from the code? Read the ticket first, then check whether the expected values in the test could only have been derived from the spec. An expected value that's just the function's current output, frozen, is a regression detector for behavior nobody validated.
• What would this test fail on? If you can't name a realistic bug that would turn it red, it's coverage theater. Tautological tests, the discount test that recomputes the discount with the same formula, are a recurring AI pattern precisely because they always pass.
• What's mocked away? A test that mocks the rule it claims to verify proves the mock works. Generated suites lean on mocks heavily because mocks make tests deterministic, and determinism looks like quality.
• Are the awaits real? An assertion inside an unawaited promise never runs, and the test passes vacuously. This one ships constantly because it's green from day one.

None of this requires heroics, it requires treating the test half of the diff as code under review rather than evidence about the other half. The moment tests stopped being written by a second mind, they stopped being a second opinion.

It's trainable, like the rest of reviewing

Deceptive tests are one of the nine failure categories on DiffDojo, and the challenges there are exactly the patterns above: tautological suites, mocked-away rules, assertions that never execute, all inside realistic PRs where everything is green and the verdict is yours to defend. Your profile then tells you whether test-reading is one of your blind spots, which for most reviewers it quietly is. Today's PR is free, no signup.

Read next: AI code reviewers miss about half the bugs, and nobody benchmarks the human catching the rest