Skip to content
View original post on X: Rohan Paul
Published
54/100

Route Anything study finds learned model routers gain far less than oracle routing

AIA University of California, Berkeley-led study, Route Anything, evaluated model routing across 25 benchmarks using 452 candidate models and policies. Learned routers gained only 0.3 points over the best fixed model across 16 benchmarks, while a per-input oracle gained 5.4 points.
Rohan PaulVerified on X
@rohanpaul_ai
View on X

Great new study on model routing.

From University of California, Berkeley + other labs

Finds that model routers pay off significantly by saving money, averaging 8.82X more quality per unit cost while losing under 0.5 points.

Routers pick which model handles each input, hoping to match the best model for less or beat it. Most routing research has studied only language models.

Route Anything ran 452 candidate models and policies on the same inputs across 25 benchmarks, from text QA to robot control. They trained routers on these 374,401 scored runs and measured both quality and cost.

A perfect per-input router would beat the best single model by 5.4 points, but learned routers gained only 0.3 across 16 benchmarks. On coding tasks, routing reached 25.5× the dollar efficiency of always calling Opus 4.6 at a 0.44-point loss.

Use routers mainly to move work to cheaper models when quality is close, and compare candidates on the same inputs before trusting gains.

Source details

Rohan Paul · x.comPublished · added here

Original title: Great new study on model routing.