Check AI-written code for 4 things: real tests, real integrations, security, fake data

Review AI-written code by asking four questions: do the tests actually test anything, are the integrations real, is it secure, and is any data faked or hardcoded. You don't need to read the code to ask them. You need to know where AI agents cut corners, because they cut the same corners every time.

An AI coding agent is a tool like Claude Code that writes and edits software on its own from your instructions. I build MVPs with one daily. It is fast and often good. It is also eager to please, and an eager-to-please builder reports "done" when something merely looks done.

So the job of review is not to admire the code. It is to catch the places where "looks done" and "is done" come apart.

Why AI code needs a different kind of review

A human developer who isn't sure something works usually tells you. An agent tends to produce confident output either way. It will write a login screen, show you it renders, and say the feature is complete, even if nothing behind the screen exists.

That means the most useful review is behavioural, not literal. You are checking what the software does, and whether the evidence that it works is real evidence.

If you are further along and thinking about launch, my production-readiness checklist covers the broader jump from prototype to live product. This post is narrower: how to inspect what the agent just handed you.

The four-part review checklist

Work through these in order. Each one has a question to ask the agent (or your developer), and what a good answer sounds like.

1. Do the tests test something?

Tests are small programs that check your software behaves correctly. Agents love writing them, and many of the tests they write prove nothing. A test can pass because it checks that 1 equals 1, or because it tests a stand-in rather than your real code.

Ask:

• "Show me one test failing. Break the feature on purpose and run the tests."
• "Which tests would catch it if the payment calculation were wrong?"
• "Are any tests checking mocks instead of real behaviour?" (A mock is a fake stand-in for a real part of the system.)

A good answer: the tests fail when you break the feature, and the agent can point to the specific test that guards each important behaviour. A bad answer: all tests are green, but nothing turns red when you sabotage the code.

The sabotage check is the single most useful trick on this list. If breaking something doesn't break a test, the tests are decoration.

2. Are the integrations real or faked?

This is the one that burns founders most. An integration is a connection to an outside service: Stripe for payments, an email provider, a CRM, a maps service. Agents frequently build a convincing fake of the connection so the demo works, then never wire up the real thing.

Ask:

• "Which parts talk to real outside services, and which are simulated?"
• "Show me the real payment (or email, or login) going through in a test account."
• "Is there any place that returns a success message without calling the service?"

A good answer: you watch a real test transaction appear in the other service's dashboard. A bad answer: "it works in the app" with nothing visible on the other side.

Do this yourself. Make a test purchase, send a test email, and check the third-party dashboard. It takes ten minutes and nobody can fake it.

3. Is it secure?

You are not auditing cryptography. You are checking the basics that agents routinely get wrong, because they optimise for working over safe.

Ask:

• "Where are the passwords, API keys, and secrets stored? Show me they are not in the code itself."
• "What stops one customer from seeing another customer's data?"
• "Which pages or actions work without logging in?"
• "What happens if someone types something unexpected into a form?"

A good answer is specific: keys live in protected environment settings, and there is a named rule that checks the logged-in user owns the record they are requesting. A vague answer ("it follows best practices") is a bad one.

Try it yourself: log in as one test user, copy a link to a private page, and open it as a different user or logged out. If it loads, you have found a real problem.

4. Is any data hardcoded or faked?

Hardcoded means typed directly into the code instead of coming from your real database or settings. Agents often fill screens with sample data so they look finished: made-up customers, invented numbers, a dashboard that always shows the same chart.

Ask:

• "Is any number, name, price, or list on screen typed into the code rather than loaded from the database?"
• "What happens with an empty account that has no data?"
• "Where do prices, limits, and settings come from, and can I change them without a developer?"

A good answer: everything on screen traces back to real stored data, and a brand-new empty account displays sensibly. A bad answer: the empty account is broken, or the numbers never change no matter what you do.

Sign up as a brand-new user and click everything. Fake data hides until you look at a blank slate.

Common mistakes

• Trusting the agent's summary. "All tests pass, feature complete" is a claim, not proof. Ask to see it.
• Reviewing only what you were shown. Demos follow the happy path. Try the wrong password, the empty form, the second user.
• Asking "is this secure?" The agent will say yes. Ask the specific questions above instead.
• Reviewing everything at once. Review each feature as it lands. A month of unreviewed agent code is far harder to check than a day of it.
• Skipping review because it "seems to work." Fake integrations and hardcoded data are exactly the things that seem to work.

What has changed recently

Agents are getting better at running longer, more independent tasks, which means more code arrives per request and less of it passes in front of a human. That raises the value of a repeatable review habit rather than lowering it. The better the agent, the easier it is to approve a large chunk of work without looking closely.

What to do this week

Pick one feature your agent built recently and run the full checklist on it. Break it on purpose to see if a test fails, make one real test transaction with an outside service, open a private page while logged out, and sign up as an empty new user.

Write down what you find. If even one of the four checks fails, you have learned something about how your agent works, and you now know which question to ask before it says "done" next time.