Building on the existing EPD architecture, our latest improvements make disaggregation more efficient through better data transfer, cache reuse, and routing.
In our DeepSeek V4.1 Flash multi-page document benchmark at concurrency 64–128:
⚡️44–47% lower P90 TTFT
🚀6.6–9.5% higher token throughput per GPU, counting every encoder GPU.
vLLM cuts DeepSeek V4.1 Flash multi-page document TTFT by 44–47%
AIvLLM says its latest EPD disaggregation improvements, covering data transfer, cache reuse, and routing, cut P90 time to first token by 44–47% in a DeepSeek V4.1 Flash multi-page document benchmark at concurrency 64–128. The same benchmark shows token throughput per GPU rising 6.6–9.5%, counting every encoder GPU.
@vllm_project
Part of a thread · earlier post
Source details
vLLM · x.comPublished · added here
Original title: Building on the existing EPD architecture, our latest improvements make disaggregation more efficient through better data transfer, cache...