SemiAnalysis releases GDPx agentic benchmark; top model scores 12%
Article timeline
The articles in this story. Times are ET.
elvisXSemiAnalysis releases GDPx, a benchmark where the top model scores 12%AISemiAnalysis releases GDPx, a benchmark for measuring agentic capabilities across high-value economic domains. It reports that even the best model scores just 12%. Elvis Saravia says Gemini, now with Carbon, has improved strongly and that Carbon is especially strong at coordination and persistence.
Rohan PaulXSemiAnalysis releases GDPx, a benchmark testing AI agents on expert workAISemiAnalysis has released GDPx, a benchmark that tests AI agents on expert work across 11 domains. Claude Opus 5.5 leads with an average score of 12%, followed by Gemini 4 Carbon at 10.9%. Unlike OpenAI's GDPval, GDPx requires agents to search through workspaces such as a 3,983-file replica of a real acquisition, and finance and law tasks score 0 under pass-fail grading.
Ethan MollickXEthan Mollick says GDPx needs human baselines like GDPvalAIEthan Mollick welcomes GDPx as a successor to GDPval but says it needs human baselines. He reports that GPT-6 Pro flagged errors in the grading and argues that the 0% scores hide economically meaningful differences in success rates.
Sundar PichaiXSundar Pichai praises GDPx benchmark's headroom and Carbon's costAIGoogle CEO Sundar Pichai says the GDPx benchmark has high-quality results with headroom and praises Carbon's cost line. SemiAnalysis released GDPx to measure agentic capabilities across valuable domains, where the best model scores just 12%.
Varun MohanXVarun Mohan praises a new eval where top scores sit below 15%AIVarun Mohan says he likes new evals where the highest score stays under 15%, pointing to the jump from Argon to Carbon and the cost-effectiveness of Gemini models. He expects the leaderboard to change substantially within a few months given the pace of progress. The post is a reply to SemiAnalysis's release of GDPx, an agentic capabilities benchmark on which the best model scores 12%.
SemiAnalysisXGDPx benchmark shows top model scores just 12% on agentic tasksAISemiAnalysis releases GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. The best model scores just 12%, or 1 of 7 tasks.
- SemiAnalysisBlogPickSemiAnalysis launches GDPx, a benchmark covering high-value frontier AI domains
AISemiAnalysis announces GDPx, a benchmark of agentic tasks across domains it says will generate the most frontier AI revenue, including chip design, cybersecurity, biology, and finance. Its tasks run in realistic environments built with 14 specialized data partners, and it says the top model scores 12%. The article contrasts this design with OpenAI's GDPval, which it says gives models all context in the prompt and relies on rubric grading.