vLLM maps Muse Spark Open serving support across NVIDIA, AMD, and TPU

AISummary

vLLM says its NVIDIA work includes fused norm, gating, and residual kernels, plus fused MNNVL collectives and an FP32-norm-to-TF32-router path.

It adds that AMD MI355X supports FP8 and MXFP4 with TP and PP on ROCm, and that initial TPU support uses TorchTPU. With DSpark at acceptance length 6, Muse Spark Open megakernels reach over 650 tokens/sec on 4 TPU v7 chips and 960 tokens/sec on 8 chips, versus over 270 and 350 tokens/sec without speculation.

TPU code access is available through the team or Google.

AIWhy it matters

The post maps Muse Spark Open's serving support across NVIDIA, AMD MI355X, and TPU, showing which optimizations and throughput figures apply to each platform.

Post on XView on X

vLLM

@vllm_project

Part of a thread · earlier post

The hardware work reaches across the serving stack.

NVIDIA's FlashInfer and vLLM optimizations include fused norm/gating/residual kernels, TEP pooler/unpooler sharding with fused MNNVL collectives, and an FP32-norm-to-TF32-router path.

AMD MI355X supports FP8 and MXFP4 with TP/PP on ROCm.

Initial TPU support uses TorchTPU. With DSpark (acceptance length = 6), Muse Spark Open megakernels reach over 650 tokens/sec on 4 TPU v7 chips and 960 tokens/sec on 8 chips; without speculation, over 270 and 350 tokens/sec. Code access is available through the team or Google.

4/6

Source: vLLM · x.comPublished Oct 15, 2:13 PM ET · added here 2:24 PM ET

vLLM maps Muse Spark Open serving support across NVIDIA, AMD, and TPU

Topics

Tags