The post explains how a draft model guesses tokens that the main model verifies in one pass, a concrete method for speeding up inference that engineers can compare with existing setups.
This will make language models generate text much faster.
Normally, a model generates one token at a time.
But with this tool, a smaller helper model guesses several upcoming tokens, and the main model checks those guesses together.
When the guesses are right, you get several tokens in one pass.
They made the code available, and you can run this with vLLM and SGLang.
EAGLE-4 is here. From EAGLE-1 to EAGLE-3, we’ve kept making inference faster. EAGLE-4 is our fastest yet. 961.6 tokens/s on @Alibaba_Qwen’s Qwen3.8-27B with TokenSpeed (BF16, 4× @NVIDIA B300, TP4, one request, T=0). Github: https://github.com/eagle-hq/EAGLE-4
Santiago · x.comPublished · added here
Original title: This will make language models generate text much faster.
