StateBench: Testing how language models track state changes in long documents
StateBench is a long-context state tracking benchmark that tests how language models maintain tracking changes in entity information within long documents. The benchmark includes four difficulty levels, ranging from simple immutable assignments to complex dependency assignments. It found that cyclic and hybrid models generalize better than Transformers on direct training tasks, but cutting-edge models like Gemini 3.8 Flash, Claude Opus 5.5, and GPT-6 Sol still fail in complex state tracking with long inputs.






















