vLLM adds BF16 and NVFP4 support for NVIDIA Blackwell GPUs

AISummary

vLLM adds Day 0 support for BF16 and NVFP4 model versions, with NVFP4 offering a smaller weight footprint and low-precision execution on NVIDIA Blackwell GPUs.

NVIDIA reports over 600 output tokens/sec at 100K input tokens on Artificial Analysis agentic tasks via a Baseten endpoint. A separate SWE-bench Pro comparison reports 4× faster task completion on 4× B200 GPUs using NVFP4 at 50 TPS/user interactivity.

Post on XView on X

vLLM

@vllm_project

Part of a thread · earlier post

BF16 and NVFP4 support from Day 0.

Use BF16 as the baseline, or serve the NVFP4 version for a smaller weight footprint and low-precision execution on NVIDIA Blackwell GPUs.

NVIDIA reports over 600 output tokens/sec at 100K input tokens on Artificial Analysis agentic tasks via a Baseten endpoint.

A separate SWE-bench Pro comparison reports 4× faster task completion on 4× B200 GPUs, with all models using NVFP4 and 50 TPS/user interactivity. The chart shows the accuracy–completion-time tradeoff.

3/4

Source: vLLM · x.comPublished Oct 15, 11:50 AM ET · added here 12:00 PM ET

vLLM adds BF16 and NVFP4 support for NVIDIA Blackwell GPUs

Topics