Skip to content
Trending storyDeveloping

Ant Group and vLLM open-source Humming, a quantized inference kernel library

2 articles2 sourcessince Oct 20Last article 37m ago ·

Overview

AISummary of 2 articles

Ant Group and the vLLM community are open-sourcing Humming, a kernel library for quantized inference that supports NVIDIA GPUs from Turing onward and the T-Head Zhenwu 810E, according to a vLLM blog post.

The post says Humming outperforms vLLM Marlin, FlashInfer and DeepGEMM across many workloads. In a DeepSeek-V4.1-Flash test on eight H20 GPUs, switching MoE execution to Humming W4A8 raised output throughput by about 14% to 43%.

A vLLM post on X gives the same 14–43% end-to-end MoE throughput gain and adds that kernel speedups reach up to 5x, citing the linked benchmarks. The blog states the package is about 500 KB with simple dependencies. Starting with vLLM 0.29.0, CUDA installations include Humming automatically, so no separate install is needed.

Written by AI from the articles below · updated Oct 21, 12:24 AM ET

Check the sources:

Latest developmentOct 21, 12:17 AM ET
vLLM and Ant Group open-source Humming, a quantization kernel library

Article timeline

The articles in this story. Times are ET.

Oct 21
  1. vLLMOfficial
    vLLM and Ant Group open-source Humming, a quantization kernel library

    AIvLLM and Ant Group open-sourced Humming, a single kernel library for quantized inference across formats, GPUs and workloads. The post reports kernel speedups of up to 5x and end-to-end MoE throughput gains of 14–43%, with benchmarks in the linked blog.

Oct 20
  1. vLLM BlogOfficial
    Ant Group and vLLM open-source Humming, a quantized inference kernel library

    AIAnt Group and the vLLM community are open-sourcing Humming, a kernel library for quantized inference that supports NVIDIA GPUs from Turing onward and the T-Head Zhenwu 810E. The source says Humming outperforms vLLM Marlin, FlashInfer, and DeepGEMM across many workloads, and reports a DeepSeek-V4.1-Flash test on eight H20 GPUs where switching MoE execution to Humming W4A8 raised output throughput by about 14% to 43%.

Heat Trend

Not enough continuous observations to show a trend yet.