Skip to content
Trending storyDeveloping

Artificial Analysis Coding Agent Index v2.0 switches to FrontierCode, Claude Opus 5.5 leads

4 articles4 sourcessince Oct 22Last article

Overview

AISummary of 4 articles

Artificial Analysis launches v2.0 of its Coding Agent Index, scoring models with their agent harnesses across 290 tasks in software engineering, terminal use, and repository Q&A, with the three benchmarks weighted equally.

The update replaces DeepSWE v1.1 with FrontierCode v1.1 from Cognition. Claude Code with Claude Opus 5.5 (max) ranks first at 61.7, according to Artificial Analysis. Meng Shao says the mean score for the same 21 configurations fell from 54.8 to 47.4 under the new version.

Cognition then releases FrontierCode 2.0, which it says uses more than 20 open-source maintainers and 55% more repositories. Cognition says the top model, Claude Opus 5.5, scores 45.5%, down from 54.6% on FrontierCode 1.1. Cognition says the benchmark is far from saturated. Artificial Analysis has not stated whether the Index will adopt FrontierCode 2.0.

Written by AI from the articles below · updated Oct 27, 1:42 PM ET

Check the sources:

Developments

2 developments

  1. Oct 27, 1:00 PM ET · 2 articles
    Cognition releases FrontierCode 2.0 coding benchmark
    Cognition Blog (Devin, Windsurf): Cognition releases FrontierCode 2.0, a coding benchmark where top model scores 45%
  2. Oct 22, 8:17 PM ET · 2 articles
    Artificial Analysis launches Coding Agent Index v2.0 with FrontierCode v1.1
    Artificial Analysis: Artificial Analysis launches Coding Agent Index v2.0 with FrontierCode v1.1

Article timeline

The articles in this story. Times are ET.

Oct 27
  1. CognitionOfficial
    Cognition launches FrontierCode 2.0, a coding benchmark where top model scores 45%

    AICognition introduces FrontierCode 2.0, a coding benchmark that tests whether a model's pull request should be merged, not just whether it meets a spec. The benchmark uses more diverse tasks, larger scopes, higher mergeability standards, and deliberately underspecified prompts that require model judgment. The top model scores 45%, which Cognition says shows the benchmark is far from saturated.

    Image from @cognition's post
  2. Cognition Blog (Devin, Windsurf)OfficialPick
    Cognition releases FrontierCode 2.0, a coding benchmark where top model scores 45%

    AICognition introduces FrontierCode 2.0, a coding benchmark built with more than 20 open-source maintainers across 55% more repositories. The top-performing model, Claude Opus 5.5, scores 45.5%, down from 54.6% on FrontierCode 1.1. Thoroughness criteria, which test unstated correctness requirements, account for the largest share of blockers and have the highest failure rate at every reasoning effort level.

Oct 23
  1. meng shaoX
    Artificial Analysis Coding Agent Index v2.0 ranks Claude Opus 5.5 first at 61.7

    AIArtificial Analysis released v2.0 of its Coding Agent Index, which evaluates models with their agent harnesses across 290 tasks. Claude Code with Claude Opus 5.5 (max) leads at 61.7, and FrontierCode v1.1 replaces DeepSWE. The mean score for the same 21 configurations fell from 54.8 to 47.4 under the new version.

    Image from @shao__meng's post
Oct 22
  1. Artificial AnalysisOfficial
    Artificial Analysis launches Coding Agent Index v2.0 with FrontierCode v1.1

    AIArtificial Analysis launched v2.0 of its Coding Agent Index, which now covers 290 tasks across software engineering, terminal use, and repository Q&A, with the three benchmarks weighted equally. FrontierCode v1.1 from Cognition replaces DeepSWE v1.1, and Claude Code with Claude Opus 5.5 (max) leads at 61.7. The source says the Index will keep updating with more evaluations and agents.

    Image from @ArtificialAnlys's post

Heat Trend

Current comparable heat
19
Peak comparable heat
29Oct 27, 3:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.