vLLM tests prefill-decode split on 12 GB300 GPUs in pre-release prototype
Overview
vLLM reports that its prefill-decode (P/D) prototype ran on 12 GB300 GPUs, connecting DEP4 prefill to DEP8 decode with FP8, NIXL and a 384 GiB cache store.
The test recorded 514,784 prefix-hit tokens (91.24%), and two 1,000-request OCR runs scored 90.4% and 89.4% with zero request errors, according to vLLM's post on X.
The test ran before release and rebase, and the P/D pull request remains open, so the final build may differ.
Written by AI from the articles below · updated Oct 14, 11:29 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
vLLM@vllm_projectOfficialvLLM prototypes prefill-decode split on 12 GB300 GPUs with FP8AIvLLM's prototype connects DEP4 prefill to DEP8 decode on 12 GB300 GPUs using FP8, NIXL and a 384 GiB cache store. The test recorded 514,784 prefix-hit tokens (91.24%) and two 1,000-request OCR runs scored 90.4% and 89.4% with zero request errors. The test ran before release and rebase, and the P/D pull request remains open.

Heat trend
Not enough continuous observations to show a trend yet.