The P/D prototype connected DEP4 prefill to DEP8 decode on 12× GB300 with FP8, NIXL and a 384 GiB cache store. It recorded 514,784 prefix-hit tokens (91.24%); two 1,000-request OCR runs scored 90.4% and 89.4%, with zero request errors.
This was a pre-release, pre-rebase test. The P/D PR remains open; check the final build separately.
5/6
vLLM prototypes prefill-decode split on 12 GB300 GPUs with FP8
AISummary
vLLM's prototype connects DEP4 prefill to DEP8 decode on 12 GB300 GPUs using FP8, NIXL and a 384 GiB cache store. The test recorded 514,784 prefix-hit tokens (91.24%) and two 1,000-request OCR runs scored 90.4% and 89.4% with zero request errors. The test ran before release and rebase, and the P/D pull request remains open.
Post on XView on X
vLLMVerified on X
@vllm_project
Part of a thread · earlier post
Source: vLLM · x.comPublished