Verifying AI Output
AI output is a first draft. Your job is to prove it works in your repo, not in a hypothetical one.
Pre-Merge Checklist
Run through this list for every AI-generated change:
| Step | Command / action | Pass criteria |
|---|---|---|
| Read the diff | git diff | Only intended files changed; no mystery edits |
| Lint | npm run lint or equivalent | Zero new errors |
| Typecheck | tsc --noEmit or build | No type errors |
| Unit tests | npm test | All pass; new code has meaningful assertions |
| Build | npm run build | Production build succeeds |
| Imports | Search new imports | Every path resolves; no phantom packages |
| Env vars | Grep process.env | New vars documented and set locally |
| Security scan | Review SQL, auth, secrets | No injection, hardcoded keys, or eval |
| Manual smoke test | Run the feature in browser/CLI | Behavior matches the request |
Models often add tests that mock everything and assert nothing useful. Read test bodies, not just green checkmarks.
Diff Review Tips
- Start with file list — reject changes to config, lockfiles, or CI unless you asked for them
- Check deletions — agents sometimes remove "unused" code that other modules import
- Watch formatting-only PRs — large style changes hide logic bugs
- Compare to the prompt — if the diff does more than requested, trim it
Verify Dependencies
Before npm install anything the model suggested:
npm view some-package-name version
Check download counts, last publish date, and whether you already have an equivalent dependency.
Runtime Verification
For web features:
- Test error states (empty input, network failure, 401)
- Test with realistic data volume, not just
"test"strings - Check accessibility if UI changed (keyboard nav, labels)
When to Reject and Re-Prompt
Reject the entire change if:
- More than one attempt still fails the same test
- The model introduces a new library for a one-line fix
- Security issues appear in auth, payments, or user input handling
Re-prompt with: the error output, the file you want changed, and what not to touch.