Replit releases ViBench 2.0, a benchmark for AI models building full software products
Overview
Replit releases ViBench 2.0, a benchmark in which AI models build 17 software products from an empty project, adding 241 feature requests and passing 544 test plans.
Replit says the best models, GPT-6.1 Sol, GPT-6 Astra and Opus 5.5, pass all test plans in only 49 to 65% of their builds.
Fireworks, Replit's partner for open-weight model inference and training, runs the official results and added ViBench 2.0 to its Specialized Intelligence Index (SII). Fireworks reports GPT-6.1 Sol leading at 65% pass@1 for about $12 per build.
Written by AI from the articles below · updated Oct 28, 3:03 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- Replit BlogOfficialViBench 2.0 benchmark tests whether AI models can build full software products
AIReplit releases ViBench 2.0, a benchmark in which models build 17 products from an empty project, adding 241 feature requests and passing 544 test plans. The best models, GPT-6.1 Sol, GPT-6 Astra and Opus 5.5, pass all test plans in only 49 to 65% of builds. Fireworks runs the official results, with GPT-6.1 Sol leading at 65% pass@1 for about $12 per build.
Heat Trend
Not enough continuous observations to show a trend yet.