# OpenBMB study finds post-training agents rarely revise their initial strategy

> Original title: An agent that can run a full post-training pipeline almost never reconsiders its initial strategy.

- Source: OpenBMB
- Published: 2026-10-26T13:00:00.000Z
- Bad Influence: https://badinfluence.news/items/moqopts51etyvegla3w3yr3mb
- Original: https://x.com/OpenBMB/status/2114703594347725005

## Summary

OpenBMB analyzed 1,338 complete post-training trajectories from PostTrainBench across 7 benchmarks, 4 base models, and 3 agent CLIs. Agents improved average scores from 33.34 to 48.55 through execution changes, but only 2.1% of 3,557 adjacent experiment pairs switched strategy. A single mid-run human sentence switching strategy raised GSM8K by 17.44 points and HumanEval by 10.80 points.

## Why it matters

The source separates execution from strategy and shows that agents rarely revise their initial post-training approach, a finding that clarifies where agent capability falls short.

## Post

An agent that can run a full post-training pipeline almost never reconsiders its initial strategy.
We analyze 1,338 complete training trajectories from PostTrainBench — 7 benchmarks, 4 base models, Claude Code / Codex CLI / OpenCode, 10h on one H100 — and split capability into EXECUTION vs STRATEGY.

1️⃣ Qualified executor, locked strategy. Agents improve over base models on every benchmark; with experience, avg score goes 33.34 → 48.55. ALL execution-level suggestions were adopted, but NONE of the strategy-level suggestions (e.g., switching to RL, adding SFT warmup) were adopted. Only 2.1% of 3,557 adjacent experiment pairs actually switch strategy. Defaults follow the agent, not the task: Claude Code starts full-parameter SFT 71.9% of the time; Codex CLI starts PEFT/LoRA 89.5%.

2️⃣ Experience and compute don't unlock it. Extra experience and memory raise execution but never leave the locked strategy; more reasoning tokens buy finer execution rather than better direction — on AIME 2025, ~14M extra tokens solve just 1 more problem.

3️⃣ A mid-run human guidance wins everywhere. At the same checkpoint, same remaining budget, one sentence switching to the alternative strategy beats autonomous continuation: GSM8K +17.44pp, HumanEval +10.80pp, AIME 2025 +6.67pp.

The missing ability isn't executing a new strategy — it's the trigger to spontaneously restart strategy decisions mid-trajectory.

📄 Paper: https://arxiv.org/abs/2608.19072
#AIAgent #PostTraining #LLM #RLHF #GRPO #PPO #RSI #AgentBenchmark #PostTrainBench #THUNLP

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHVjQYPIbgAA5M4D.jpg&mode=thumb&exp=1793404800&sig=c52918bf41089fcb)