# StateBench tests how language models track state changes over long documents

> Original title: StateBench: Testing how language models track state changes over long documents

- Source: Ai2 (Allen Institute for AI)
- Published: 2026-11-03T08:00:00.000Z
- Bad Influence: https://badinfluence.news/items/zaypunfp3fayeunnwbdxg0ik6
- Original: https://allenai.org/blog/statebench

## Summary

Ai2 introduces StateBench, a benchmark that tests how well language models track entity state changes in long documents across four difficulty levels. The team reports that recurrent and hybrid models generalize much farther than transformers when trained directly on these tasks. It also finds that frontier models, including Gemini 3.8 Flash, Claude Opus 5.5, and GPT-6 Sol and Luna, still break down on harder state tracking as inputs grow longer.