Ant Group and vLLM open-source Humming, a quantized inference kernel library with Zhenwu 810E support
AIAnt Group and the vLLM community have open-sourced Humming, a just-in-time compiled GEMM kernel library for quantized LLM inference, under Apache 2.0. The release says it fully supports T-Head's Zhenwu 810E accelerator, and starting with vLLM 0.29.0 it is included in CUDA installs of vLLM. The performance figures come from the teams' own benchmarks and have not been independently verified.
AIWhy it matters.
The article ties a kernel library's release to specific hardware support beyond NVIDIA, which matters for teams weighing quantized inference options. Its performance numbers come from the teams' own tests.
Loading source posts…
Source details
Pandaily · contact@pandaily.com (Pandaily) · pandaily.comPublished · added here
Original title: Ant Group and vLLM Open-Source Humming, Quantized Inference Kernels With Zhenwu 810E Support