Nebius's Ryan Hanrui Wang on KV-cache management and agent latency and cost

Nebius AI's Ryan Hanrui Wang argues that moving from fast tokens to fast agents depends on three choices: KV-cache management, scheduling, and model selection.

LMSYS Org, which shared his remarks on X, reports him saying these decisions determine how much context can be reused, how fast agents respond, and what production agents cost to run.

The report is a single X post summarizing his argument and gives no figures, benchmarks, or product details behind the claims.