2026-07-20 · 6 min read

How to review AI-generated code: a guide for the human in the loop

Search this topic and you'll mostly find vendors selling an AI that reviews the AI's code, which is a fine thing to run and they do catch real things, but the approval button is still yours and "the other model didn't object" is not a review. This is about the human half: what AI-generated code actually gets wrong in practice, and how to get better at seeing it before you click approve.

Why AI bugs are hard to catch

A tired human writes code that looks tired: weird names, half-finished thoughts, commented-out experiments, a TODO that says "fix this later" in a function that clearly needs fixing now. The mess itself is a signal, and experienced reviewers have learned to slow down wherever the code smells off, because messy code is usually where the bugs live.

A model writes code that looks finished. Clean formatting, confident naming, a helpful docstring sitting on top of a function that does the wrong thing. Every bug wears the same calm face as the correct code around it, and the usual reviewer instinct, the one that says "this part smells off, read it twice," has nothing to grab onto because nothing smells off. Everything smells like a tutorial example.

So the question that works on human PRs, "does this look reasonable?", stops working on AI PRs, because AI code always looks reasonable. The question that replaces it is: what is this code assuming, and did anyone actually promise that? Models fill every gap in the spec with a plausible guess, and the guesses are where the bugs live.

Try it on a real diff

Before the list, a test. This is the kind of PR an assistant produces when asked to "import users in bulk and report when done". Would you approve it?

If you caught it, good. Now imagine it on line 240 of a 600-line PR, formatted beautifully, between two changes that are completely fine.

The six failure modes worth checking every time

1. Async that doesn't actually wait

The example above is the canonical version of this, but it shows up in a lot of shapes: loops with async callbacks where nobody awaits the loop itself, promises that get created and then silently dropped on the floor, chains of .then() calls where the test function never returns the promise. The symptom to hunt for is always the same: the function reports success before the work is provably done, and nothing in the code structure makes that obvious, because there's an await in there somewhere and it looks like it's doing its job.

2. APIs that don't exist

Models autocomplete plausible method names from patterns in their training data, not from the library you actually have installed. You end up with a utility call that reads exactly like the ten real ones next to it in the file, except the library never shipped it. It survives review precisely because it reads well; it dies at runtime the moment someone actually calls the path that exercises it. If a method call is load-bearing and you don't personally remember it existing, check the docs. It takes thirty seconds, and it's the cheapest catch you'll ever make in a review.

3. Almost the spec, but not the spec

Ask for "retry failed payments up to three times with backoff" and you may get code that retries three times, with backoff and everything, on every error including the permanent ones that should fail immediately. The code is adjacent to the requirement, defensible in isolation, and wrong. This is where models are most convincingly incorrect, because the code genuinely does something sensible; it's just not the thing that was asked for. The habit that catches this is rereading the requirement after you've read the diff, then checking whether each clause in the spec actually survived into the implementation rather than getting approximated into something close enough to pass a glance.

4. The happy path is the only path

Models write for the demo case, the one where the list has three items and every API call returns 200 and every field is present and well-formed. What they don't write for is the empty list, the malformed row, the API that returns 500, the null that the spec explicitly allows but nobody thought to test. The question to ask of every function in the diff is: what's the ugliest input this can legally receive, and what happens when it does?

5. State, cleanup, and time

Stale closures, effects without cleanup functions, race conditions when two requests overlap because the user clicked twice. These are the bugs that require you to simulate time in your head, to imagine what happens when this code runs and then runs again before the first run finished, and that kind of reading is exactly the kind nobody does when the diff "looks clean" and the reviewer has eleven other PRs in the queue. If the change involves anything concurrent or long-lived, walk one interleaving by hand before you approve, even if it means spending an extra two minutes on a diff that looks perfectly fine.

6. Security defaults nobody asked for

Object lookups without ownership checks, user content rendered as raw HTML, secrets interpolated into client-side code, endpoints that serve data without verifying who's asking. The model isn't hostile; it's obliging. It builds the endpoint you asked for and skips the authorization check you didn't explicitly mention, because you didn't mention it and the model's job is to do what you asked. Anything in the diff that touches auth, money, or user content gets the slow read, and the question to ask is always the same: who else can reach this, and what can they do with it that the author didn't intend?

Building the reflex

Knowing the list is the easy part, and if knowing it were enough then this article would be all you need. The hard part is running these checks at 4pm on the eleventh PR of the day, when the code looks fine and the tests are green and merging is one click away and you have a standup in ten minutes. That's not knowledge anymore; that's a reflex, and reflexes only come from reps with feedback, from doing the thing enough times that the right question fires automatically instead of requiring conscious effort.

You can build the feedback loop yourself if you want: have an assistant generate small PRs with a planted bug, review them cold, then check whether you found the plant. It works, though writing realistic plants is its own skill and self-graded reps tend to go easy on you in exactly the places where you need them to be hardest. Or use ours: DiffDojo serves one realistic AI-written PR a day across these exact failure modes, you leave your comments and your verdict, and it grades you against a canonical review that shows you what you caught, what you missed, and whether any of your comments were noise. Including the days when the diff is clean and the right answer is to approve without inventing problems, because restraint is part of the skill too. The daily PR is free, no signup.

Either way, keep the question taped to your monitor: what is this code assuming? The model won't tell you, because from the model's perspective its assumptions are facts. That's the job now.