ReplaySSM caches SSM inputs instead of state to speed hybrid-model decoding

AITri Dao says SSMs had good asymptotics but lacked inference tools like speculative decoding and quantization that Transformers get. He credits caching SSM inputs instead of the state, which separates output computation from state updates during decoding. The quoted post says ReplaySSM runs in production in GLM-5.3-Flash and Kimi-K3 and ships in TensorRT-LLM, vLLM, FlashInfer, and SGLang, with decoding up to 1.47x faster and speculative decoding up to 2.1x faster.

AINVIDIA's technical blog presents ReplaySSM, a method that changes how state-space model layers store state to speed decoding in hybrid models. The post says hybrid models such as NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash interleave SSM layers with a few attention layers, using Mamba-2, Gated DeltaNet, or Kimi Delta Attention as the SSM layer. The source excerpt provided does not give benchmark results, so the speed gains are not quantified here.