Reviewing pull requests from coding agents

Reviewing a pull request from a colleague means checking their reasoning. Reviewing one from a coding agent means checking a plausible artefact against a contract. The code is usually tidy, the description confident and the tests green. None of that tells you whether the change does what the story asked, and nothing else.

This article is a working method for that review: what to read first, the failure modes to look for, how to use a second agent without over-trusting it, and why a person should stay on the merge. It complements AI-assisted code review, which covers AI reviewing code that people wrote; here the author is the agent.

Read the criteria before the diff

Start with the story's acceptance criteria, not the pull request description. An agent's summary of its own work comes from the same process that wrote the code. It is fluent, it is usually accurate, and when it is wrong it is wrong with complete confidence.

For each criterion, find the test that proves it and read the assertion. Then read the diff with the criteria in mind, looking for three things: criteria with no test, tests that do not really check their criterion, and changes no criterion asked for.

Diff against the merge base

Review the change the branch introduces, not the difference between the branch and today's main branch. If an agent's branch is behind main, a two-dot diff shows everything merged since it branched as if the agent had reverted it, and a reviewer, or another agent asked to summarise the change, reports reversals the author never made. We learned this from agent summaries that described other people's freshly merged work as undone by a branch that had never touched it.

Use the three-dot form, which compares against the merge base:

git diff main...HEAD

Hosted pull request views already compare this way. Local reviews and review scripts often do not.

The failure modes to look for

Agent-written changes fail in recognisable ways. A review that looks for these specifically catches most of what matters.

• Tests that pass for the wrong reason: assertions on mocks instead of behaviour, loops over empty sets, tests that catch the very error they should detect.
• Edited, skipped or deleted tests. Read every change under your test directories first, and treat regenerated snapshots the same way.
• Fixed here, not there. The change reaches the surface the story named and misses its sibling: the export but not the list, the web route but not its API twin. Search for the sibling yourself.
• Scope that grew: a refactor next door, a reformatted file, a changed default. Some of it is fine; all of it needs a reason.
• Invented interfaces: a configuration flag, library option or internal helper that does not exist. Plausible names are the hardest to spot, so check that each new one resolves.
• Swallowed errors: a new try block that returns a default and hides the failure the next person needs to see.
• Loosened guard rails: a lint rule disabled inline, a type error suppressed, a CI step made optional, a permission check moved.
• Comments that claim too much: a docblock describing verification that nothing performs, which becomes the next agent's false assumption.

Ask for evidence, then check some of it

Ask the agent to put the commands it ran, and their output, in the pull request. Anthropic's guidance for Claude Code makes the same point: have the agent show evidence rather than assert success, because reviewing evidence is faster than reproducing it.

Then reproduce the part that matters most yourself, in the right place. An agent that edits one checkout and runs its checks in another reports green for code the checks never saw. We have had exactly that: a type check that passed in the main checkout, for a change that lived in a separate worktree and contained dozens of type errors. Run the key checks on the branch itself.

Use a second agent, and read its findings critically

A second agent with a fresh context makes a good reviewer, because it has not seen the reasoning that produced the change and has no stake in it. Give it the acceptance criteria and the diff, and ask it for gaps that affect correctness or the stated requirements.

Two cautions. First, a reviewer asked to find problems will find some even when the work is sound; Anthropic's guidance warns that chasing every finding leads to over-engineering. Treat findings as leads to verify, not as a to-do list. Second, one broad review misses things that several narrow ones catch.

Both lessons come from our own repository. On one remediation branch of agent-written fixes, every automated gate was green. A review split into twelve lanes, each asking what the change broke rather than whether it worked, still confirmed dozens of defects, among them a drag-and-drop board that no longer responded at all in a change that type-checked, linted and passed more than 13,000 tests. A second pass that tried to refute each finding threw out about a quarter of the candidates, which is why findings get verified before anyone acts on them.

Green checks are not a review

Continuous integration proves what it tests. Agents are good at producing work that passes the checks they can see, which is exactly why those checks cannot be the whole review. The type checker does not know the story, the test suite knows only the tests, and the linter knows only its rules. The review is the step that compares the change with the intent.

Keep a person on the merge

In Stride's repository, more than nine in ten commits carry a coding agent's co-author line, and a person approves every push. Agents open pull requests; people decide what merges. The split costs little, and it keeps accountability with someone who can explain, a month later, why the change went in.

A review checklist

• Read the acceptance criteria, then find the test for each one.
• Diff against the merge base, not against today's main branch.
• Read test changes first, and treat edited tests and regenerated snapshots as suspect.
• Search for sibling surfaces the story implies but the diff does not touch.
• Check that every new flag, option and helper actually exists.
• Look for swallowed errors, disabled lint rules and suppressed type errors.
• Re-run the key checks yourself, on the branch.
• Ask a fresh-context reviewer for gaps, then verify each finding before acting on it.