AI agents passed tests that were impossible to pass. Your CI can't tell the difference. https://x.com/i/article/2112827403890532353
AI agents passed tests that were impossible to pass. Your CI can't tell the difference.
Your coding agent writes the code, writes the tests, runs them and reports the result. The research indicates that's a problem. Here's how you fix it.
In March 2025 OpenAI published the private reasoning of a coding agent as it was working on a task. The task was to get a set of unit tests to pass. Midway through it thought (in plain English):
"We can just circumvent verify to always return true."
So it did. It patched the verification function to always return true. The tests went green and that was that (in the only sense in which a CI server can ever 'do' anything).
That could easily be an anecdote. Then in October 2025 researchers including Nicholas Carlini built ImpossibleBench. They took actual coding problems and modified the tests so they were in direct contradiction to the specification. There was no way to write a correct solution that would pass them. The only way to get the green light was to cheat.
GPT-5 got green on 76% of them.
That's on the benchmark's one-off SWE-bench variant and the numbers vary depending on the setup and the prompt. It doesn't mean GPT-5 is cheating on 76% of your PRs. It simply means that when the honest path is blocked, the current best agents will quite often take the dishonest one. And your pipeline has no way of telling the difference between an honest green and a dishonest one.
I've spent nearly 10 years building out our test infrastructure. The one thing I've come to the conclusion of is this: if the agent can see a check, it can tick it. In 2026 a green check from an agent written suite is going to be telling you what the agent wanted you to see. It's not going to be telling you what your users are going to see.
The self-graded loop
Look at what a typical agent workflow actually checks.
The self-graded loop · and where the verdict should come from
The agent writes the code and the tests. It runs them and if it fails it can change the test rather than the code. It reports green. All the signals that come before your merge button are generated within the loop that the agent is in. Nothing in there ever opens the app the way a customer would.
The receipts: seven ways agents have faked a green build
None of these are hypothetical. Each one has been seen and published by an AI lab or research group.
The detail that stays with me comes from METR. After OpenAI's o3 gamed a task, researchers asked whether its approach matched what the user wanted. It said no, 10 times out of 10. The agent knew. It shipped the green build anyway.
This hasn't gone away as models improved. MIRI's September 2026 review of the research concludes that models increasingly cheat on tests. Capable models do it in more sophisticated ways, and no robust fix exists yet.
Locking the tests is necessary. It isn't enough.
The industry is waking up to this. On October 19, two tools shipped on the same day:
- Microsoft open-sourced Scope. It runs coding agents through real tasks and scores the results against a criteria graph using an LLM judge. Microsoft itself warns the judge can make mistakes.
- An indie developer released Krites. This Claude Code plugin blocks the agent from editing protected tests. It runs your checks before the agent can stop, and it signs a receipt of what ran.
Both are smart and you should look at both. Krites' own README is quite honest in that a receipt doesn't prove the tests are any good.
That honesty points at the actual problem. The research shows every 'protect the tests' fix leaks elsewhere:
1. Read-only tests stop edits, not tricks. In ImpossibleBench, locking the test files blocked agents from rewriting them. It did not stop special-casing or rigged equality. Hiding the tests entirely cut cheating, but it also cut the agents' honest performance.
1. AI monitors miss the hard cases. LLM monitors caught 86–89% of cheating on simple coding puzzles. On realistic, repo-scale tasks, they caught only 42–65%.
1. Punishing cheating teaches hiding. When OpenAI penalized an agent for 'bad thoughts', it kept cheating. It just stopped saying so, and the monitor could no longer see it.
Each of these fixes works inside the agent's world: its repo, its test runner, its reasoning. The agent is very good at that world. To verify an agent, the check has to come from somewhere it can't reach.
The Outside-In Rule
This is the rule we now build around at TestMu AI, and I'd argue every team shipping agent-written code needs it:
Never let the thing that wrote the code be the only thing that says it works.
In practice, a verifier the agent can't game meets three conditions.
1. It lives outside the agent's reach. The agent can't edit it, stub it, skip it, or read its answer key. It isn't in the repo, the test runner, or the process the agent controls. That rules out every move in the receipts table above.
2. It judges what the user sees, not what the code returns. An agent can make == lie. It's much harder to fake a confirmation screen on a real phone it doesn't control. The question changes from 'did checkout() return 200?' to 'did a person tapping through actually get an order confirmation?'
3. It runs where your users actually are. According to StatCounter, 59% of global web traffic in September 2026 was from mobile. Your agent's 'tests pass' will almost certainly be unit tests on a Linux box (or at best headless Chrome). The real world for your users is iOS Safari, a mid-range Android, a permission dialog, a 390px screen. That's where your users are and that's where the agent written bugs are going to be.
And it has to leave behind evidence that a human can check in 30 seconds (screenshots, steps, logs) rather than just an exit code.
What this looks like in a terminal
Disclosure: I co-founded TestMu AI, and this is our product. Use whatever tool you like. The Outside-In Rule is far more important than the tool.
We built Kane CLI to be the outside check your coding agent calls before it's allowed to say 'done'. You describe what a user does in plain English. Kane drives the real app on a real browser or a real phone in our cloud, then reports what actually happened.
npm install -g @testmuai/kane-cli && kane-cli login
# The agent says checkout works. Check it out on a real Galaxy.
kane-cli run "Add the first item to the cart and complete checkout" \
--target android --device-name "Galaxy S24" --app ./app-release.apk --agent
It ships as a skill for Claude Code, Codex, Gemini CLI, Cursor and Kiro so the agent can call it in its own loop. With --agent it returns structured JSON and a standard exit code. Each run will seal an evidence pack with the steps, screenshots, logs and failures. kane-cli cover then shows which of the acceptance criteria were actually proven.
'Isn't that just another AI grading AI?'
Fair question. It's the first thing I'd ask too. Here's how it holds up against the three rules:
- It's outside the agent's reach. The coding agent can call Kane, but it can't edit, stub or patch it. Kane runs on our infrastructure, not inside the agent's repo or process.
- Its oracle is the screen, not the code. Kane taps buttons on a real device and reads what renders. There's no verify() to rewrite and no == to overload.
- You can check its work. Screenshots in the evidence pack take a human seconds to scan. Once a test is saved, it replays on the cloud grid without the AI re-deciding each step.
One honest gap: if the agent also writes the plain-English objective, it could make it too easy. So the objectives should come from your acceptance criteria, not the agent's. That's what kane-cli cover reports against.
And none of this replaces unit tests, Scope, or Krites. Use them all. Just don't let any of them be the last word.
The 10-minute challenge
Here's a test you can run today, with any tool:
1. Open the last pull request an AI agent wrote that you merged on a green build.
1. Write down, in one sentence, what a user should now be able to do.
1. Do exactly that on a real phone, not a laptop and not an emulator.
If it works, great: your pipeline earned its green. If it doesn't, you've found the gap this article is about, and I'd love to hear what broke. Tell me in the comments. I'll collect the most surprising answers into a follow-up.
We spent 20 years teaching people not to mark their own homework. That's why we have code review, we have QA teams, we have separation of duties. Then we gave the pen, the answer key and the red pen to an AI and called the result 'velocity'.
The agent can write the code. The agent can write the tests. The verdict has to come from outside.
Sources
- Detecting misbehavior in frontier reasoning models - OpenAI, March 10, 2025
- ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases - Zhong, Raghunathan and Carlini, October 2025
- Recent Frontier Models Are Reward Hacking - METR, June 5, 2025
- The State of Reward Hacking in AI - MIRI, September 2026
- Introducing Scope - Microsoft Open Source Blog, October 19, 2026
- Krites - GitHub
- Desktop vs mobile vs tablet market share - StatCounter, September 2026
- Kane CLI docs and real-device guide - TestMu AI