Skip to content
Trending storyDeveloping

Artificial Analysis finds tool use lifts VIALS-AA scores for several models

2 articles1 sourcesince Nov 11Last article

Overview

AISummary of 2 articles

Artificial Analysis reports that tool use sharply raises VIALS-AA scores for several models.

On 23 cell counting and quantification tasks, Fable's pass^3 rate falls from 65% with tools to 9% without, Claude Opus 5.5 (max) from 57% to 4%, and Claude Sonnet 5.5 (max) from 65% to 13%. GPT-6 Astra (max) falls from 61% to 35%, and GPT-6.1 Sol (max) from 61% to 39%. On 23 phylogenetics tasks, Astra passes 78% with tools and 83% without.

In a separate rerun of the VIALS-AA visual reasoning benchmark in one-shot format with no tools, Artificial Analysis says average scores rise from about 25% to 47% when tools are used. Claude Fable 5.2 (max) drops from 67% to 44% without tools, below Claude Haiku 5.5 (max) at 49% with tools. Artificial Analysis reports Haiku costs $0.07 per task, about a tenth of Fable's $0.73 cost without tools.

Written by AI from the articles below · updated Nov 11, 7:03 PM ET

Check the sources:

Developments

2 developments

  1. Nov 11, 6:46 PM ET · 1 article
    Artificial Analysis compares VIALS-AA scores with and without tools
    Artificial Analysis: Artificial Analysis finds tools lift VIALS-AA scores, letting Claude Haiku 5.5 beat Fable 5.2
  2. Nov 11, 6:46 PM ET · 1 article
    Artificial Analysis reports tool-use effects on VIALS-AA model scores
    Artificial Analysis: Tools lift VIALS-AA cell counting pass rates for Fable and Claude models

Article timeline

The articles in this story. Times are ET.

Nov 11
  1. Artificial AnalysisOfficial
    Tools lift VIALS-AA cell counting pass rates for Fable and Claude models

    AIArtificial Analysis reports that on 23 cell counting and quantification tasks in VIALS-AA, Fable's pass^3 rate falls from 65% with tools to 9% without, Claude Opus 5.5 (max) from 57% to 4%, and Claude Sonnet 5.5 (max) from 65% to 13%. GPT-6 Astra (max) falls from 61% to 35% and GPT-6.1 Sol (max) from 61% to 39%. On 23 phylogenetics tasks, Astra passes 78% with tools and 83% without.

    Image from @ArtificialAnlys's post
  2. Artificial AnalysisOfficial
    Artificial Analysis finds tools lift VIALS-AA scores, letting Claude Haiku 5.5 beat Fable 5.2

    AIArtificial Analysis reran its VIALS-AA visual reasoning benchmark without tools and found tool use nearly doubles average scores, from about 25% to 47%. Without tools, Claude Fable 5.2 (max) drops from 67% to 44%, falling below Claude Haiku 5.5 (max) at 49% with tools. Haiku costs $0.07 per task, about a tenth of Fable's $0.73 cost without tools.

    Image from @ArtificialAnlys's post

Heat Trend

Current comparable heat
15
Peak comparable heat
19Nov 11, 10:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.