Reflection reports that DFlash2 speeds up Beam's decode path by 1.72× to 3.03× across GSM8K, MATH500, HumanEval, MBPP and MT-Bench at concurrency 1, 16 and 32.
On GSM8K the speedups are 3.03×, 2.51× and 2.75× at those concurrency levels, measured in a disaggregated prefill/decode setup with prefill time excluded. Reflection also reports better throughput with K=7, although the drafter was trained with block size 16.
Part of a thread · earlier post
DFlash2 speeds up Beam's decode path.
Reflection reports 1.72–3.03× decode-only speedups across GSM8K, MATH500, HumanEval, MBPP and MT-Bench at concurrency 1, 16 and 32. GSM8K reaches 3.03× / 2.51× / 2.75×, respectively.
Measured in a disaggregated prefill/decode setup; prefill time is excluded.
Separately, Reflection reports better throughput with K=7, even though the drafter was trained with block size 16.
Source: vLLM · x.comPublished · added here
