vLLM adds bi-directional KV transfer to cut TTFT in multi-turn agent workloads

vLLM's bi-directional KV transfer lets prefill nodes reuse KV blocks retained by decode nodes across conversation turns, instead of recomputing the full history.

In AWS benchmarks on a 1P1D deployment with EC2 P5en instances, TTFT fell up to 1.6x for Qwen3-32B and up to 3.3x for GLM-5-FP8 at long prompts.

Results depend on workload and configuration, and the feature requires vLLM v0.21.0 or later, NixlConnector, and a KV-cache-aware router.