StreamEval is now on ModelScope for evaluating interactive streaming video generation across 15 to 300-second sessions.
🤖 https://modelscope.ai/datasets/StreamEval/StreamEval
📄 https://modelscope.ai/papers/2610.28402
🎬 300 long-form interactive sessions: Each 15 to 300-second case delivers multimodal requests over time and checks both state changes and content preservation.
🧩 Five tasks, ten metrics: Covers avatar dialogue, avatar performance, world generation, instruction editing, and reference editing across execution, consistency, visual fidelity, and audiovisual coordination.
📊 Visual quality is not enough: Across 37 models, strong visuals and identity retention did not reliably mean better instruction following or sustained state consistency.
🔓 Fully open and updated monthly: Benchmark data, evaluation code, protocols, and scoring rules are public, with monthly leaderboard updates for new and updated models.
📜 Benchmark data: CC BY 4.0. Evaluation code: Apache 2.0. Third-party editing footage requires attribution.