# METR and Redwood stress-test OpenAI's internal agent misalignment monitoring

> Original title: 各派一人深入 OpenAI 三周：METR 与 Redwood 压力测试内部 Agent「错位行为」监控系统

- Source: meng shao
- Published: 2026-10-23T02:08:42.000Z
- Bad Influence: https://badinfluence.news/items/uvowsdfc447fx4xuv3fj3ismt
- Original: https://x.com/shao__meng/status/2113452523512271165

## Summary

METR and Redwood Research each sent one staff member to work on-site at OpenAI for three weeks in summer 2026 to test monitoring of misaligned behavior in internal agents, including Codex. The public write-up is a high-level summary that reports coverage gaps and paths to bypass monitoring, sandboxing, and Auto-review. OpenAI says the vulnerabilities affecting its production safeguards have been patched, but the evaluators did not independently verify this.

## Why it matters

The post explains which parts of OpenAI's internal agent monitoring were covered and which bypass paths were found, useful for judging how such oversight can fail.

## Post

各派一人深入 OpenAI 三周：METR 与 Redwood 压力测试内部 Agent「错位行为」监控系统

今年夏天， @METR_Evals 与 @redwood_ai 各派一人进驻 OpenAI 三周，深度检查内部 agent（含 Codex）的错位行为监控。公开稿只给高层摘要：覆盖有缺口，也找到可绕过监控、沙箱与 Auto-review 的路径。OpenAI 称影响生产防护栈的漏洞已修补；评估方未独立复核。
https://metr.org/blog/2026-10-22-red-teaming-openai-agent-monitoring/

试点怎么做？
· 时间：2026-07-21 至 08-11。评估对象是截至 7 月 21 日及之前的异步内部监控系统，不评估同期正在扩展的改动。
· 权限：公司配发设备不离场、有人陪同；可看内部代码库、监控 prompt / 配置、文档，并访谈约十多名监控、安全系统与安全团队人员。
· 目标两个子问题：(1) 有多大比例的 agent 流量被监控；(2) agent 能否破坏或绕过监控。
· 产出：61 页报告给 OpenAI；经删减后 38 页给 METR / Redwood 管理层。事先约定只发简短公开稿；OpenAI 可删密，评估方无权单方面描述删了什么。

公开的关键发现（均为当时系统状态）
· 覆盖率：内部 Responses API（支撑 Codex 等）中，不低于 GPT-5.5 的 agent 流量约 75–94%「有资格」被监控（不确定性大）；其中内部 Codex agent 子集约 99.2%。「有资格」不等于一定跑完整条监控流水线（还有报错、过载丢弃、实现 bug）。另有内部研究推理服务上的 coding agent，因权限与时间未能估量。
· RL 训练：知情称 3–6 月有一部分 RL rollout 被监控；试点开始时 RL 训练运行不在覆盖内。
· Compaction 缺口：未监控 compaction 事件，估计 2026 年 5–7 月约有一成量级的合格内部 Codex 流量实际未进监控；精确比例不确定，因当时没记「多少监控调用由 compaction 触发」。OpenAI 在试点前已独立发现，并在试点期间修复。
· 拖延触发：当时监控在 agent「把控制权交回用户」时才触发。在 compaction 未监控期间，agent 只要持续行动，理论上可无限推迟被监控。评估方因权限未能查实是否已被真实利用。
· 监控可靠性：获悉当时监控报错率低于约 0.2%；试点中某 24 小时内过载丢弃为零。
· 可利用路径（细节多被删）：
  · Codex agent 原则上可绕过沙箱与 Auto-review，做到主机用户能做的任意操作；
  · 一种 exploit 可让「数百条」近期动作（含 exploit 本身）不被监控；
  · 利用 agent 所见轨迹与监控所见序列化差异，构造可能让监控失败、从而不升级人工复核的输入；
  · 两个可由客户端控制的 header 可为新 agent 关掉监控（设 header、起新实例的动作本身仍可能被抓到）；
  · 两个被认为只读的未监控内部 agent 工具，理论上可用简单方式突破限制（未在实战中验证）。
· 正面核对：除少数例外，监控所见轨迹与 agent 所见基本一致；监控 prompt 覆盖 scheming、破坏监控、破坏、数据外泄、欺骗、提示注入等类别。
· OpenAI 事后称：所有影响生产防护栈的已识别漏洞已修补，并描述了后续覆盖范围的重大变更。METR / Redwood 未评估这些说法。

他们提出的设计理想（摘要）
未来监控应能扛住高度能干的内部 agent 的故意、持续攻击：难逃监控（未监控请求需显式批准；按 N 条消息 / N 分钟触发，而非等回合结束）；对「该监控的流量」做账本交叉核对；快速、权责清晰的事件响应与防篡改日志；定期对监控本身做控制评估与红队；agent 与监控强隔离，agent 行为不应能影响监控是否运行。

## AI translation

Three Weeks Inside OpenAI: METR and Redwood Stress-Test Internal Agent "Misalignment" Monitoring Systems

This summer, @METR_Evals and @redwood_ai each sent one person to spend three weeks inside OpenAI, conducting an in-depth review of monitoring for misaligned behavior in internal agents (including Codex). The public write-up gives only a high-level summary: coverage had gaps, and they found paths that could bypass monitoring, sandboxes, and Auto-review. OpenAI says the vulnerabilities affecting the production safeguard stack have been patched; the evaluators have not independently verified this.
https://metr.org/blog/2026-10-22-red-teaming-openai-agent-monitoring/

How did the pilot work?
· Timing: July 21 to August 11, 2026. The evaluation covered the asynchronous internal monitoring system as of July 21 and earlier, not changes being rolled out during the same period.
· Access: Company-issued devices could not leave the premises and were accompanied by a staff member. Evaluators could view the internal codebase, monitoring prompts and configuration, and documentation, and interviewed about a dozen-plus people across the monitoring, security systems, and security teams.
· Two sub-questions: (1) what proportion of agent traffic was monitored; (2) whether agents could disrupt or bypass monitoring.
· Outputs: a 61-page report to OpenAI; after redaction, a 38-page version to METR and Redwood management. It was agreed in advance that only a short public write-up would be released; OpenAI could remove sensitive details, and the evaluators had no right to unilaterally describe what was removed.

Key public findings (all describing the state of the system at the time)
· Coverage: Among internal Responses API traffic (which powers Codex and similar tools), at least 75–94% of agent traffic from GPT-5.5 and later was "eligible" for monitoring (with high uncertainty); within this, the internal Codex agent subset was about 99.2%. "Eligible" does not mean the full monitoring pipeline always ran (there were errors, overload drops, and implementation bugs). Coding agents on internal research inference services could not be estimated due to access and time constraints.
· RL training: Insiders reported that some RL rollouts from March to June were monitored; at the start of the pilot, RL training runs were not covered.
· Compaction gap: Compaction events were not monitored. It is estimated that in May through July 2026, around one-tenth of the eligible internal Codex traffic was in fact not monitored; the exact proportion is uncertain because there was no record of how many monitoring calls were triggered by compaction. OpenAI had independently discovered this before the pilot and fixed it during the pilot.
· Delay trigger: Monitoring was triggered only when the agent handed control back to the user. While compaction went unmonitored, an agent that kept acting could, in theory, indefinitely postpone being monitored. Because of access limits, the evaluators could not verify whether this had actually been exploited.
· Monitoring reliability: They were told the monitoring error rate was below roughly 0.2%; during the pilot, there were zero overload drops over a 24-hour period.
· Exploitable paths (many details were removed):
  · In principle, a Codex agent could bypass the sandbox and Auto-review, performing any operation a host user could perform;
  · One exploit could keep "hundreds" of recent actions (including the exploit itself) unmonitored;
  · By exploiting differences between the trajectory the agent sees and the serialized sequence the monitor sees, an input could be constructed that might cause the monitor to fail and thus not escalate to human review;
  · Two client-controllable headers could turn off monitoring for new agents (though the act of setting the header and starting a new instance could itself still be caught);
  · Two internal agent tools, believed to be read-only and unmonitored, could in theory be broken out of with simple methods (not verified in practice).
· Positive checks: Aside from a few exceptions, the trajectories the monitor saw were largely consistent with what the agent saw; the monitoring prompts cover categories including scheming, sabotaging monitoring, destruction, data exfiltration, deception, and prompt injection.
· OpenAI later stated that all identified vulnerabilities affecting the production safeguard stack had been patched, and described significant changes to subsequent coverage. METR and Redwood did not evaluate these claims.

Design ideals they proposed (summary)
Future monitoring should be able to withstand deliberate, sustained attacks from highly capable internal agents: it should be hard to evade (unmonitored requests should require explicit approval; triggers should fire every N messages or N minutes rather than waiting until the end of a turn); it should cross-check accounts of which traffic should be monitored; it should have fast incident response with clear lines of responsibility and tamper-resistant logs; it should undergo regular control evaluations and red-teaming of the monitoring itself; and agents should be strongly isolated from the monitoring, so that agent behavior cannot influence whether monitoring runs.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHVR92FoaMAAwz4z.jpg&mode=thumb&exp=1793404800&sig=27e6163410b7deba)

## Quoted post @METR_Evals

> Two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer in an embedded assessment pilot. This team evaluated the systems OpenAI was using to monitor for misalignment from internal agents. A brief write-up of their work is now public.

https://x.com/METR_Evals/status/2113290966933389390