PyTorch adds optional FlyDSL kernel backend for AMD MI350-series GPUs
Overview
PyTorch now lets users on AMD MI350-series GPUs (gfx950) run FlyDSL kernels through existing operator APIs for dense and grouped GEMM, MXFP8/MXFP4 scaled GEMM, RMSNorm, TopK, and attention, without calling custom kernels.
The backend is optional. GEMMs use it through torch.compile, and eligible RMSNorm and TopK calls dispatch automatically in eager mode when it is available and enabled. Unsupported inputs keep the existing PyTorch implementations.
PyTorch's blog reports kernel-level geometric-mean speedups of 1.10x to 4.82x against ATen or Triton, depending on operator and baseline. It reports end-to-end vLLM whole-request speedups of 1.13x for BF16, 2.04x for MXFP8, and 3.01x for MXFP4, varying by model and concurrency.
Written by AI from the articles below · updated Oct 21, 2:20 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- PyTorch BlogOfficialPickPyTorch adds FlyDSL kernels for AMD MI350-series GPUs
AIPyTorch now lets users on AMD MI350-series GPUs run FlyDSL kernels through existing APIs for dense and grouped GEMM, MXFP8/MXFP4 scaled GEMM, RMSNorm, TopK, and attention. Reported kernel-level geometric-mean speedups range from 1.10x to 4.82x against ATen or Triton, depending on the operator and baseline. End-to-end vLLM whole-request speedups reach 1.13x for BF16, 2.04x for MXFP8, and 3.01x for MXFP4, varying by model and concurrency.
Heat Trend
Not enough continuous observations to show a trend yet.