Skip to content
View original post on X: Artificial AnalysisOfficial
Published
34/100

Tools lift VIALS-AA cell counting pass rates for Fable and Claude models

AIArtificial Analysis reports that on 23 cell counting and quantification tasks in VIALS-AA, Fable's pass^3 rate falls from 65% with tools to 9% without, Claude Opus 5.5 (max) from 57% to 4%, and Claude Sonnet 5.5 (max) from 65% to 13%. GPT-6 Astra (max) falls from 61% to 35% and GPT-6.1 Sol (max) from 61% to 39%.
Artificial AnalysisVerified on XOfficial
@ArtificialAnlys
View on X

Part of a thread · earlier post

Tools matter most for counting tasks, which need an exact tally of many small objects. With tools, models can crop, pull pixel-level intensities and further manipulate images to separate objects and count them in code. Without them, they have to estimate from what they see.

On the 23 cell counting and quantification tasks in VIALS-AA, Fable drops from a 65% pass^3 rate with tools to just 9% without. Claude Opus 5.5 (max) falls from 57% to 4%, and Claude Sonnet 5.5 (max) falls from 65% to 13%.

GPT-6 Astra (max) and GPT-6.1 Sol (max) hold up better: Astra passes 61% with tools and 35% without, and Sol falls from 61% to 39%. Some domains depend less on tools: on the 23 phylogenetics tasks, Astra passes 78% with tools and 83% without.

Source details

Artificial Analysis · x.comPublished · added here

Original title: Tools matter most for counting tasks, which need an exact tally of many small objects. With tools, models can crop, pull pixel-level inte...