Skip to content
Trending storyDeveloping

Artificial Analysis publishes Coding Agent Index comparing agent costs and benchmarks

2 articles1 sourcesince Oct 13Last article 1h ago ·

Overview

AISummary of 2 articles

Artificial Analysis has published a Coding Agent Index that compares coding agents on per-task cost and benchmark results, linking to an interactive index and a methodology page.

In the index's first reported comparison, Artificial Analysis says Claude Sonnet 5.5 scores 68 at max effort and 63 at xhigh effort in Claude Code. Per task, xhigh costs $3.33 against $14.19 at max, a 77% cut for 5.5 fewer Index points. Max generates 4.8 times as many output tokens and takes 3.4 times as many turns. Artificial Analysis says max improves all three benchmark scores, with the largest gain on Terminal-Bench 4.0, rising from 58.1% to 66.2%.

Written by AI from the articles below · updated Oct 13, 8:50 PM ET

Check the sources:

Developments

2 developments

  1. Oct 13, 8:44 PM ET · 1 article
    Artificial Analysis publishes Coding Agent Index with task costs and benchmark results
    Artificial Analysis: Artificial Analysis launches Coding Agent Index with task costs and benchmarks
  2. Oct 13, 8:44 PM ET · 1 article
    Artificial Analysis finds Claude Sonnet 5.5 scores 68 at max effort versus 63 at xhigh on its Coding Agent Index
    Artificial Analysis: Sonnet 5.5 leads the Coding Agent Index at max effort, xhigh costs far less

Article timeline

The articles in this story. Times are ET.

Oct 13
  1. Artificial AnalysisOfficial
    Artificial Analysis launches Coding Agent Index with task costs and benchmarks

    AIArtificial Analysis has published a Coding Agent Index that compares coding agents on task costs and benchmark results. The post links to the interactive index and a methodology page describing how the agents were benchmarked.

  2. Artificial AnalysisOfficialPick
    Sonnet 5.5 leads the Coding Agent Index at max effort, xhigh costs far less

    AIArtificial Analysis says Sonnet 5.5 scores 68 on its Coding Agent Index in Claude Code at max effort, versus 63 at xhigh. Per task, xhigh costs $3.33 against $14.19 at max, a 77% cut for 5.5 fewer Index points. Max generates 4.8 times as many output tokens and takes 3.4 times as many turns, improving all three benchmark scores, with the largest gain on Terminal-Bench 4.0 rising from 58.1% to 66.2%.

    Image from @ArtificialAnlys's post

Heat trend

Not enough continuous observations to show a trend yet.