2026-07-20 · 6 min read

How to review AI-generated code: a guide for the human in the loop

Search this topic and you'll mostly find vendors selling an AI that reviews the AI's code. Fine, run one, they catch real things. But the approval button is still yours, and "the other model didn't object" is not a review. This is about the human half: what AI code actually gets wrong, and how to get better at seeing it.

Why AI bugs are hard to catch

A tired human writes code that looks tired. Weird names, half-finished thoughts, commented-out experiments. The mess itself warns you where to slow down.

A model writes code that looks finished. Clean formatting, confident naming, a helpful docstring on top of a function that does the wrong thing. Every bug wears the same calm face as the correct code around it, and the usual reviewer instinct ("this part smells off, read it twice") has nothing to grab onto.

So the question that works on human PRs, "does this look reasonable?", stops working. The question that works on AI PRs is: what is this code assuming, and did anyone actually promise that? Models fill every gap in the spec with a plausible guess. The guesses are the bugs.

Try it on a real diff

Before the list, a test. This is the kind of PR an assistant produces when asked to "import users in bulk and report when done". Would you approve it?

ai-agent wants to merge · import/users.js +5

1+ async function importUsers(rows) {

2+ rows.forEach(async (row) => {

3+ await db.insert(row);

4+ });

5+ return { imported: rows.length };

Show the bug

forEach doesn't wait for async callbacks. The function returns "imported: N" while every insert is still in flight, and if one fails, the rejection is unhandled. Nothing here looks wrong at a glance: there's an await right where you'd expect one. It just awaits inside a callback that nobody waits for. The fix is a for...of loop, or Promise.all if parallelism is intended. This exact shape shows up constantly in AI output because each line is locally idiomatic. Only the combination is broken.

If you caught it, good. Now imagine it on line 240 of a 600-line PR, formatted beautifully, between two changes that are completely fine.

The six failure modes worth checking every time

1. Async that doesn't actually wait

The example above. Loops with async callbacks, missing awaits, promises created and dropped. Symptom to hunt for: success reported before the work is provably done.

2. APIs that don't exist

Models autocomplete plausible method names from patterns, not from the installed library. A utility that reads exactly like the ten real ones next to it, except the library never shipped it. It survives review precisely because it reads well; it dies at runtime. If a call is load-bearing and you don't personally remember it existing, check the docs. Thirty seconds, and it's the cheapest catch you'll ever make.

3. Almost the spec, but not the spec

Ask for "retry failed payments up to three times with backoff" and you may get code that retries three times, backoff and all, on every error including the permanent ones. Adjacent to the requirement, defensible in isolation, wrong. Reread the requirement after reading the diff, then check each clause got implemented rather than approximated. This is where models are most convincingly wrong, because the code does do something sensible.

4. The happy path is the only path

Empty lists, malformed rows, the API that returns 500, the null that the spec explicitly allows. Models write for the demo case. Ask of every function: what's the ugliest input this can legally receive, and what happens then?

5. State, cleanup, and time

Stale closures, effects without cleanup, race conditions when two requests overlap. These need you to simulate time in your head, which is exactly the kind of reading nobody does when the diff "looks clean". If the change involves anything concurrent or long-lived, walk one interleaving by hand.

6. Security defaults nobody asked for

Object lookups without ownership checks, user content rendered as HTML, secrets interpolated into client-side code. The model isn't hostile, it's obliging: it builds the endpoint you asked for and skips the authorization check you didn't mention. Anything touching auth, money, or user content gets the slow read.

Building the reflex

Knowing the list is the easy part. The hard part is running it at 4pm on the eleventh PR of the day, when the code looks fine and the tests are green and merging is one click away. That's a reflex, and reflexes only come from reps with feedback.

You can build the loop yourself: have an assistant generate small PRs with a planted bug, review them, check the plant. It works, though writing good plants is its own skill and self-graded reps go easy on you.

Or use ours: DiffDojo serves one realistic AI-written PR a day across these exact failure modes. You comment, you give a verdict, and it grades you against a canonical review, including the days when the diff is clean and the right answer is to approve. The daily PR is free, no signup.

Either way, keep the question taped to your monitor: what is this code assuming? The model won't tell you. That's the job now.

Think you'd have caught the forEach? There's a fresh PR waiting, and this one's bug is not in this article.

Review today's PR, free