ModelScope releases StreamEval benchmark for interactive streaming video generation

ModelScope releases StreamEval, a benchmark for evaluating interactive streaming video generation.

It contains 300 sessions lasting 15 to 300 seconds, scores five tasks across ten metrics, and ModelScope says testing 37 models found strong visual quality did not reliably predict better instruction following or sustained state consistency.

The benchmark data is licensed CC BY 4.0 and the evaluation code Apache 2.0, with third-party editing footage requiring attribution. ModelScope says the data, code, protocols and scoring rules are public, with leaderboard updates each month for new and updated models.