Qwen Team releases BabelArena, a multilingual agent benchmark across 23 languages

AIQwen Team introduces BabelArena, an open-source multilingual agent benchmark covering 23 languages, with the dataset and paper linked in the post. It reports that no single model leads every benchmark.

AIWhy it matters.

The post reports error patterns in low-resource languages, such as tool-use failures and extra token use, that accuracy scores alone would miss.

ModelScopeOfficial

@ModelScope2022

View on X

In the Bible, humans tried to build the Tower of Babel but failed when language split them. Today, AI Agents are trying to "build" tasks across 23 languages—and facing the exact same barrier. 🧱🗣

Qwen Team introduces BabelArena, a new open-source multilingual Agent benchmark on 🧵👇
🔗 Dataset: https://modelscope.ai/datasets/Qwen/BabelArena
📄 Paper: https://modelscope.ai/papers/2609.23490

🏆 No single model dominates all benchmarks.
⚠ Low-resource languages show distinct failure modes: high rates of tool-use and control-flow errors (not just answer quality).
📈 Agents consume up to ~2x more tokens in low-resource languages without more interaction rounds.
🗣 Language consistency drops in structured-output tasks, with switches overwhelmingly pointing to English.

Source details

ModelScope · x.comPublished Oct 25, 11:52 PM ET

Original title: In the Bible, humans tried to build the Tower of Babel but failed when language split them. Today, AI Agents are trying to "build" tasks ...

Qwen Team releases BabelArena, a multilingual agent benchmark across 23 languages

AIWhy it matters.

Topics

Tags