Surge AI releases DAYJOB: Tax benchmark; top AI agent passes 10.8% of tasks
Overview
Surge AI releases DAYJOB: Tax, a benchmark of 50 professional tax environments across eight areas of tax practice.
Scores are pass@1 at a 95% rubric threshold. Claude Sonnet 5.5 leads with a 10.8% pass rate, followed by Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%, according to a Hacker News post summarizing the benchmark.
The benchmark tests agents on assignments that need specialized expertise, complex materials, and decisions the initial instructions do not fully specify. The source says agents often failed to flag issues the client never raised, such as a residence-based sale exclusion that could expire in August 2027.
Written by AI from the articles below · updated Oct 28, 6:07 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- Hacker News · AI (150+ points)BlogSurge AI's DAYJOB: Tax benchmark finds AI agents pass fewer than 11% of tax assignments
AISurge AI introduces DAYJOB: Tax, a benchmark of 50 professional tax assignments across eight practice areas. Claude Sonnet 5.5 leads with a 10.8% pass rate at a 95% rubric threshold, followed by Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%. The source says agents often failed to flag issues the client never raised, such as a residence-based sale exclusion that could expire in August 2027.
Heat Trend
Not enough continuous observations to show a trend yet.