Skip to content
Trending storyDeveloping

AMD lands megakernel decode support in upstream vLLM; CUDA has no draft PRs yet

1 article1 sourcesince Oct 12Last article 1h ago ·

Overview

AISummary of 1 article

AMD has landed megakernel decode support in upstream vLLM, available on ROCm but not yet on CUDA, according to SemiAnalysis.

The technique reduces kernel launches by 10x to improve high-interactivity decoding. SemiAnalysis says CUDA vLLM has no draft pull requests for upstream megakernel support yet.

Written by AI from the articles below · updated Oct 12, 11:28 PM ET

Check the sources:

Latest developmentOct 12, 11:00 PM ET
AMD lands megakernel decode support in upstream vLLM before NVIDIA

Article timeline

The articles in this story. Times are ET.

Oct 12
  1. SemiAnalysisX
    AMD lands megakernel decode support in upstream vLLM before NVIDIA

    AIAMD has landed megakernel decode support in upstream vLLM, a feature available on ROCm but not yet on CUDA, according to SemiAnalysis. The technique reduces kernel launches by 10x to improve high-interactivity decoding, and CUDA vLLM has no draft PRs for upstream megakernel support yet.

    Image from @SemiAnalysis_'s post

Heat trend

Not enough continuous observations to show a trend yet.