Ant Group and vLLM open-source Humming, a quantized inference kernel library
AIAnt Group and the vLLM community are open-sourcing Humming, a kernel library for quantized inference that supports NVIDIA GPUs from Turing onward and the T-Head Zhenwu 810E. The source says Humming outperforms vLLM Marlin, FlashInfer, and DeepGEMM across many workloads, and reports a DeepSeek-V4.1-Flash test on eight H20 GPUs where switching MoE execution to Humming W4A8 raised output throughput by about 14% to 43%.
Loading source posts…
Source details
vLLM Blog · Ant Group and the vLLM Community · vllm.aiPublished · added here
Original title: Announcing Humming: Fast, Flexible Kernels for Quantized Inference