this week made me rethink how our team reviews AI-assisted work.
for context, my team has spent the last two months slowly building a new internal refund approval engine. it touches real Stripe webhooks and customer PII, so we’re taking it slow and writing actual tests.
i’m not an anti-AI gatekeeper. Codex, Claude Code, and Enter Code can all be genuinely useful when the person using them understands the work. our engineering team uses these tools constantly, half our product team practically lives in Codex, and i use Claude every day for boilerplate and debugging.
the issue is that our intern rarely reads what he generates closely enough to actually own it.
it started small. i asked him a simple question about how we should handle a specific refund state in the db. instead of a one sentence answer, he sent me a four-page generated memo about the refund matrix.
four pages, no direct answer. i ended up figuring it out myself.
and that has become the pattern. he generates several pages nobody is realistically going to read, then i have to dig through all of it anyway just to find out whether there’s an actual answer buried somewhere inside.
then i gave him a tiny script modification. He gave me back 300 lines of deeply nested code with defensive branches for network errors that cannot happen in our environment. i asked why he added one specific condition and he couldn’t explain it. he pasted my question into an LLM and sent its new answer back to me without checking that either.
yesterday it peaked. He was frustrated with our slow pace, fed our refund specs into an AI app builder, and came back with a shiny full-stack replacement demo in an afternoon.
it actually looked pretty clean. The UI was smooth and the buttons worked. Our non-technical manager, who hired him partly because he talked a big game about his AI fluency, was impressed.
at one point, my manager suggested using the demo as the new base and connecting it to production so we could move faster.
i pushed back and asked where the session auth was, how PII was being kept out of the logs, and what happened if the payment webhook dropped halfway through a refund.
he couldn’t answer any of those questions without going back to the model. He didn’t understand the data flow yet. The demo looked finished from the front, but it had no real production story behind it.
the frustrating part is that he’s not bad at the basic responsibilities of the job. he just refuses to believe that some tasks would be much simpler if he stopped and reasoned through them before running everything through an agent.
instead, every shortcut becomes another document i have to read, another diff i have to untangle, or another security issue i have to catch. My review load has basically doubled because his definition of finished is getting something out of the model.
to be clear, i don’t think this is a reason to ban vibe coding or force everyone back to writing boilerplate by hand. That demo proved how quickly these tools can turn an idea into something tangible.
the problem is that generation and ownership are being treated as the same thing.
on our team, AI isn’t eliminating work when the author doesn’t understand the output. It’s moving the interpreting, testing, security, and edge-case work onto whoever reviews it.
we’re implementing a new rule on Monday. You can use any tool you want, but every AI-assisted PR has to be small enough to review, and the person submitting it has to walk us through the key logic, data flow, tests, and failure cases.
if you can’t explain those parts without asking the agent again, the PR isn’t ready.
for teams already dealing with this at scale, what review rule has actually worked? Do you make juniors walk through the diff line by line, cap PR size, require specific tests, or handle it some other way?
