StateBench tests how language models track state changes over long documents
AIHugging Face introduces StateBench, a benchmark for long-context state tracking across four difficulty levels, from immutable assignment to assignment by reference. In its tests, recurrent and hybrid models generalize much farther than transformers when trained directly on these tasks.
Loading source posts…
Source details
Hugging Face Blog · huggingface.coPublished · added here
Original title: StateBench: Testing how language models track state changes over long documents