StateBench tests how language models track state changes over long documents
AIAi2 introduces StateBench, a benchmark that tests how well language models track entity state changes in long documents across four difficulty levels. The team reports that recurrent and hybrid models generalize much farther than transformers when trained directly on these tasks.
Loading source posts…
Source details
Ai2 (Allen Institute for AI) · allenai.orgPublished · added here
Original title: StateBench: Testing how language models track state changes over long documents