Three Weeks Inside OpenAI: METR and Redwood Stress-Test Internal Agent "Misalignment" Monitoring Systems
This summer, @METR_Evals and @redwood_ai each sent one person to spend three weeks inside OpenAI, conducting an in-depth review of monitoring for misaligned behavior in internal agents (including Codex). The public write-up gives only a high-level summary: coverage had gaps, and they found paths that could bypass monitoring, sandboxes, and Auto-review. OpenAI says the vulnerabilities affecting the production safeguard stack have been patched; the evaluators have not independently verified this.
https://metr.org/blog/2026-10-22-red-teaming-openai-agent-monitoring/
How did the pilot work?
· Timing: July 21 to August 11, 2026. The evaluation covered the asynchronous internal monitoring system as of July 21 and earlier, not changes being rolled out during the same period.
· Access: Company-issued devices could not leave the premises and were accompanied by a staff member. Evaluators could view the internal codebase, monitoring prompts and configuration, and documentation, and interviewed about a dozen-plus people across the monitoring, security systems, and security teams.
· Two sub-questions: (1) what proportion of agent traffic was monitored; (2) whether agents could disrupt or bypass monitoring.
· Outputs: a 61-page report to OpenAI; after redaction, a 38-page version to METR and Redwood management. It was agreed in advance that only a short public write-up would be released; OpenAI could remove sensitive details, and the evaluators had no right to unilaterally describe what was removed.
Key public findings (all describing the state of the system at the time)
· Coverage: Among internal Responses API traffic (which powers Codex and similar tools), at least 75–94% of agent traffic from GPT-5.5 and later was "eligible" for monitoring (with high uncertainty); within this, the internal Codex agent subset was about 99.2%. "Eligible" does not mean the full monitoring pipeline always ran (there were errors, overload drops, and implementation bugs). Coding agents on internal research inference services could not be estimated due to access and time constraints.
· RL training: Insiders reported that some RL rollouts from March to June were monitored; at the start of the pilot, RL training runs were not covered.
· Compaction gap: Compaction events were not monitored. It is estimated that in May through July 2026, around one-tenth of the eligible internal Codex traffic was in fact not monitored; the exact proportion is uncertain because there was no record of how many monitoring calls were triggered by compaction. OpenAI had independently discovered this before the pilot and fixed it during the pilot.
· Delay trigger: Monitoring was triggered only when the agent handed control back to the user. While compaction went unmonitored, an agent that kept acting could, in theory, indefinitely postpone being monitored. Because of access limits, the evaluators could not verify whether this had actually been exploited.
· Monitoring reliability: They were told the monitoring error rate was below roughly 0.2%; during the pilot, there were zero overload drops over a 24-hour period.
· Exploitable paths (many details were removed):
· In principle, a Codex agent could bypass the sandbox and Auto-review, performing any operation a host user could perform;
· One exploit could keep "hundreds" of recent actions (including the exploit itself) unmonitored;
· By exploiting differences between the trajectory the agent sees and the serialized sequence the monitor sees, an input could be constructed that might cause the monitor to fail and thus not escalate to human review;
· Two client-controllable headers could turn off monitoring for new agents (though the act of setting the header and starting a new instance could itself still be caught);
· Two internal agent tools, believed to be read-only and unmonitored, could in theory be broken out of with simple methods (not verified in practice).
· Positive checks: Aside from a few exceptions, the trajectories the monitor saw were largely consistent with what the agent saw; the monitoring prompts cover categories including scheming, sabotaging monitoring, destruction, data exfiltration, deception, and prompt injection.
· OpenAI later stated that all identified vulnerabilities affecting the production safeguard stack had been patched, and described significant changes to subsequent coverage. METR and Redwood did not evaluate these claims.
Design ideals they proposed (summary)
Future monitoring should be able to withstand deliberate, sustained attacks from highly capable internal agents: it should be hard to evade (unmonitored requests should require explicit approval; triggers should fire every N messages or N minutes rather than waiting until the end of a turn); it should cross-check accounts of which traffic should be monitored; it should have fast incident response with clear lines of responsibility and tamper-resistant logs; it should undergo regular control evaluations and red-teaming of the monitoring itself; and agents should be strongly isolated from the monitoring, so that agent behavior cannot influence whether monitoring runs.