Decision models trail language models on knowledge and reasoning but run faster and cheaper

Artificial Analysis says decision models match language models on classification and tool-calling while running faster and cheaper, but trail them on knowledge and reasoning.

Claude Haiku 5.5 (Low) scores 61 on the Decision Index, 3 points ahead of Jev 1.13, but takes 1.4 seconds per task against Jev's 0.13 seconds, and costs $184 per 1M tasks against $42.

A separate Artificial Analysis report says decision models' accuracy falls below their stated confidence at almost all probability levels on knowledge and reasoning tasks. The gap is largest on AA-Omniscience-Decision and HLE-MultiChoice. Most models are relatively well calibrated on WildGuardTest and When2Call.