METR's AI task-length benchmark doubles eight times; Claude Opus 5.5 builds the charts
Overview
Ethan Mollick says METR's AI task-length benchmark has doubled eight times, and its most recent measurement covered a task that takes humans 17 hours.
He reports that METR has since said it can no longer accurately measure AI task length with its current methods.
Mollick also says he used Claude Opus 5.5 in Claude Projects to produce the charts in his post, a job he says took minutes instead of hours. In a separate example, he says the cost of Google-Proof Q&A fell from 30 cents to four hundredths of a cent per correct answer.
Written by AI from the articles below · updated Oct 20, 12:53 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- One Useful Thing (Ethan Mollick)BlogMETR's AI task-length benchmark doubled eight times, and Claude Opus 5.5 built the charts
AIEthan Mollick says a 17-hour human-work task was the most recent METR measurement, after which METR said it could no longer accurately measure AI task length with its current methods. Google-Proof Q&A cost fell from 30 cents to four hundredths of a cent per correct answer, and Mollick used Claude Opus 5.5 in Claude Projects to produce his post's charts, which he says took minutes instead of hours.
Heat Trend
Not enough continuous observations to show a trend yet.