An agent that can run a full post-training pipeline almost never reconsiders its initial strategy.
We analyze 1,338 complete training trajectories from PostTrainBench — 7 benchmarks, 4 base models, Claude Code / Codex CLI / OpenCode, 10h on one H100 — and split capability into EXECUTION vs STRATEGY.
1️⃣ Qualified executor, locked strategy. Agents improve over base models on every benchmark; with experience, avg score goes 33.34 → 48.55. ALL execution-level suggestions were adopted, but NONE of the strategy-level suggestions (e.g., switching to RL, adding SFT warmup) were adopted. Only 2.1% of 3,557 adjacent experiment pairs actually switch strategy. Defaults follow the agent, not the task: Claude Code starts full-parameter SFT 71.9% of the time; Codex CLI starts PEFT/LoRA 89.5%.
2️⃣ Experience and compute don't unlock it. Extra experience and memory raise execution but never leave the locked strategy; more reasoning tokens buy finer execution rather than better direction — on AIME 2025, ~14M extra tokens solve just 1 more problem.
3️⃣ A mid-run human guidance wins everywhere. At the same checkpoint, same remaining budget, one sentence switching to the alternative strategy beats autonomous continuation: GSM8K +17.44pp, HumanEval +10.80pp, AIME 2025 +6.67pp.
The missing ability isn't executing a new strategy — it's the trigger to spontaneously restart strategy decisions mid-trajectory.
📄 Paper: https://arxiv.org/abs/2608.19072
#AIAgent #PostTraining #LLM #RLHF #GRPO #PPO #RSI #AgentBenchmark #PostTrainBench #THUNLP