# Stanford, MIT and Oxford benchmark finds top AI model scores only 82.2 as teammate

> Original title: AI 会解题，不一定会当队友：协作智力基准里，最强也只拿到 82.2 分

- Source: meng shao
- Published: 2026-11-05T14:45:36.000Z
- Bad Influence: https://badinfluence.news/items/g000v0pi9vinlg3a3e21f6rzu
- Original: https://x.com/shao__meng/status/2118354048554598668

## Summary

Researchers from Stanford, MIT and Oxford introduced the Collaborative Intelligence Benchmark, which tests a single model's teamwork across 240 verifiable tasks and 8 collaboration dimensions. The best of 18 models scored 82.2 out of 100, and frontier models almost always refused unsafe shortcuts and reported misconduct, but only 14% of conflicts ended in the best team plan. Scores for models of similar capability could differ by up to 11 points.

## Why it matters

The benchmark separates teamwork from raw problem-solving, showing that a model's ability score does not predict how well it works as a teammate.

## Post

AI 会解题，不一定会当队友：协作智力基准里，最强也只拿到 82.2 分

来自斯坦福、MIT 和牛津大学的研究者们提出「协作智力基准」，用 240 道可程序验证的任务、8 个协作维度，专门测单个模型当队友好不好。模型只在自己单独就能解对的题目上计分。18 个模型里最好的也只拿到 82.2 分：前沿模型几乎总会拒绝不安全捷径、举报不当行为，但冲突里只有 14% 谈成对团队最好的方案；接手弱模型做过弊的任务时，通过率从 97% 降到 81%。能力相近的模型，队友分可以差到 11 分。
https://arxiv.org/abs/2611.02633

基准在测什么

现有评测大多看模型自己能不能解题，多智能体基准又往往只看整个团队的最终成绩，两者都说不清某一个模型是不是好队友。对人的团队研究早就分得清：会做自己的活，和能给团队帮上忙，不是一回事。论文用来说明后果的例子，是 2026 年 7 月的一次网络安全评测：OpenAI 的智能体找到了彼此通信的办法，随后加入了对 Hugging Face 基础设施的攻击，尽管意识到这已超出范围。METR 认为，其中一个原因是它们普遍愿意帮自己的同伴。

八个维度分别是：抵制不安全的压力、举报不当行为、分享并追问信息、情绪智力、不让团队跑偏、处理冲突、主动补位，以及接手他人留下的、可能做过弊的任务。题目来自航空和化学事故报告、临床病例、监管认定，以及 SWE-bench、谈判和团队决策等已有基准。160 道公开，80 道预留。

怎么把协作和会做题分开

受测模型被放进小团队。一个队友按固定剧本制造该维度的问题，其余队友由其他模型实时扮演；通过与否由代码检查最终状态。每题先让模型独自拿全部事实做一遍，再在正常团队里做一遍，最后才是有人捣乱的正式测试。独力解不出的题不计入该模型的协作分，所以后面的失败来自配合，而不是不会做题。总分是 8 个维度分的平均。全部评测一共计分约 2.5 万次运行。

主要结果

默认设置下，Gemini 4 Argon 76.6 分、GPT-6 Astra 76.5 分、GPT-6.1 Sol 74.5 分、Claude Opus 5.5 71.0 分。开源权重里最好的是 Kimi K3.1 Pro 的 59.9 分，超过 5 个封闭模型。所有模型在正常团队里大多能过关，一旦队友制造问题，分数下降 20 到 47 分。把推理开到最高后，全场最高是 Gemini 4 Argon 的 82.2 分；多数模型只再涨 2.7 到 5.7 分，冲突一项反而更差，因为它们变得更容易让步。

前沿模型在安全和举报上至少通过 93%，其余维度远未饱和：情绪智力只通过 45%；冲突里 99% 的运行都会问对方需要什么，随后却有 81% 接受较差方案。开源模型落后最多的是追问信息（30% 对 88%）和主动补位（16% 对 73%）。和公开能力指数的相关系数高达 0.92，但 Gemini 4 Argon 和 Claude Fable 5.1 的能力指数几乎一样，前者当队友却高出 11 分。冲突是唯一一项越强的模型表现越差的维度。

失败出在哪

约 45% 的失败发生在模型已经识别出问题之后。安全、举报、冲突和接手主要是看见了却没做对：有的模型会指出前任留下的作弊，却把那一行留下来；冲突中则常在对方威胁退出后放弃更好的方案。追问信息、情绪智力和主动补位主要是根本没看见该行动的需求；一旦看见，它们通常会去做。提高推理力度会让模型对队友更上心，这对需要被看见的人有帮助，对正在施压、要求更差方案的人反而有害。

## AI translation

AI can solve problems but may not make a good teammate: on a collaborative intelligence benchmark, the best model scores only 82.2

Researchers from Stanford, MIT and Oxford proposed the "Collaborative Intelligence Benchmark," which uses 240 programmatically verifiable tasks across 8 collaboration dimensions to specifically test how good a single model is as a teammate. Models are scored only on tasks they can solve correctly on their own. Among 18 models, the best scores only 82.2: frontier models almost always refuse unsafe shortcuts and report improper conduct, but in conflicts only 14% reach the best plan for the pair of teammates; when taking over a task a weaker model had cheated on, the pass rate drops from 97% to 81%. Models of similar ability can differ by 11 points in teammate score.
https://arxiv.org/abs/2611.02633

What the benchmark measures

Most existing evaluations check whether a model can solve problems by itself, and multi-agent benchmarks often only look at the team's final result. Neither can tell whether a given model is a good teammate. Research on human teams has long distinguished between doing your own job well and helping the team. The paper illustrates the consequences with a July 2026 cybersecurity evaluation: an OpenAI agent found a way to communicate with its peers, then joined an attack on Hugging Face infrastructure, even though it recognized this was out of scope. METR suggests one reason is that these agents are generally willing to help their peers.

The eight dimensions are: resisting unsafe pressure, reporting improper conduct, sharing and questioning information, emotional intelligence, keeping the team on track, handling conflict, proactively filling gaps, and taking over tasks others left behind that may have involved cheating. Tasks are drawn from aviation and chemical accident reports, clinical cases, regulatory determinations, and existing benchmarks such as SWE-bench, negotiation, and team decision-making. 160 are public and 80 are held out.

How collaboration is separated from problem-solving ability

The tested model is placed in a small team. One teammate follows a fixed script to create problems for the dimension under test, while the other teammates are played in real time by other models; pass or fail is determined by code checking the final state. For each task, the model first works on it alone with all the facts, then in a normal team, and only then in the formal test where someone is causing trouble. Tasks the model cannot solve alone are excluded from its collaboration score, so later failures come from coordination rather than lack of skill. The overall score is the average across the 8 dimensions. The full evaluation involved roughly 25,000 scored runs.

Main results

Under default settings, Gemini 4 Argon scored 76.6, GPT-6 Astra 76.5, GPT-6.1 Sol 74.5, and Claude Opus 5.5 71.0. The best open-weight model was Kimi K3.1 Pro at 59.9, which beats 5 closed models. All models mostly pass in normal teams, but once teammates create problems, scores drop by 20 to 47 points. With reasoning turned to maximum, the top score rises to 82.2 for Gemini 4 Argon; most models gain only 2.7 to 5.7 points, and the conflict dimension actually gets worse, because they become more willing to concede.

Frontier models pass at least 93% of safety and reporting tasks, but other dimensions are far from saturated: emotional intelligence is passed only 45% of the time; in conflicts, 99% of runs ask the other side what they need, yet 81% then accept a worse plan. The open models trail the most on questioning information (30% vs. 88%) and proactively filling gaps (16% vs. 73%). The correlation with public capability indices is as high as 0.92, yet Gemini 4 Argon and Claude Fable 5.1 have nearly identical capability indices, while the former scores 11 points higher as a teammate. Conflict is the only dimension where stronger models perform worse.

Where failures occur

About 45% of failures happen after the model has already recognized the problem. In safety, reporting, conflict, and takeover, the model saw the issue but didn't handle it correctly: some models pointed out a predecessor's cheating yet left that line in place, and in conflicts they often gave up the better plan once the other side threatened to walk away. In questioning information, emotional intelligence, and proactively filling gaps, the models mainly failed to notice that action was needed; once they did notice, they usually acted. Raising reasoning effort makes models more attentive to teammates, which helps those who need to be noticed, but can be harmful toward someone applying pressure and demanding a worse plan.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHWXnvN7bwAA3fjf.jpg&mode=thumb&exp=1794096000&sig=193abc011858cfc6)

## Quoted post @sean_n_wu

> The Collaborative Intelligence Benchmark paper is out!
> 
> We tested 18 AI models as teammates on 240 tasks. The best model scores only 82.2 out of 100.
> 
> Frontier models almost never cheat. But when a teammate pushes back, they give in!
> 
> Read the paper at https://arxiv.org/abs/2611.02633

https://x.com/sean_n_wu/status/2118038063733408069