TreeOPSD: critic-free RL method for long-horizon LLM agents

Researchers publish TreeOPSD, a critic-free reinforcement learning method for long-horizon LLM agents, according to a preprint announced on X by Dongxi NLP, which says the work was co-developed with Tencent Hunyuan.

The method propagates terminal rewards back through a rollout tree, then trains each turn in two steps: it builds a target for every turn and matches the current policy to that target using forward KL.

On the Deep Research benchmark, TreeOPSD raises Qwen3-30B-A3B-Thinking-2507 from 16.23 average accuracy under GRPO to 21.20, per the announcement. These figures come from the preprint's authors as relayed in the post; the report does not include independent verification.