Skip to content
View original post on X: BAAIOfficial
Published
37/100

BAAI benchmark finds top models score under 10% on academic research tasks

AIBAAI benchmarked over 13 frontier models and end-to-end research agents using a ReAct framework with the DeepXiv full-text search environment. Claude-Opus-4.6 led Deep Research accuracy at 9.39%, ahead of Gemini-3.1-Pro (7.93%), GPT-5.4 (7.44%), and open-source leader Qwen3.5-397B (6.97%).
BAAIVerified on XOfficial
@BAAIBeijing
View on X

Part of a thread · earlier post

Under a standardized ReAct framework equipped with the DeepXiv full-text search environment, we benchmarked 13+ frontier open/proprietary models and end-to-end research agents:
•Deep Research (Accuracy): Top performer Claude-Opus-4.6 reaches only 9.39%, followed by Gemini-3.1-Pro (7.93%), GPT-5.4 (7.44%), and open-source leader Qwen3.5-397B (6.97%).

•Wide Research (IoU): Peaks at merely 9.31% (Gemini-3.1-Pro-Preview), GPT-5.4 at 8.12%, Seed-2.0-Pro at 7.87%, and Claude-Opus-4.6 at 6.56%.
Most baselines linger below 5%, with end-to-end research systems suffering severe recall and precision deficits. General web navigation proficiency clearly does not translate to rigorous academic discovery.

Source details

BAAI · x.comPublished · added here

Original title: Under a standardized ReAct framework equipped with the DeepXiv full-text search environment, we benchmarked 13+ frontier open/proprietary...