Skip to content
Trending storyDeveloping

vLLM adds online-fitted acceptance estimator for Adaptive Verification in draft models

2 articles2 sourcessince Oct 13Last article 1h ago ·

Overview

AISummary of 2 articles

vLLM adds an online-fitted acceptance estimator that lets draft models without a trained confidence head use Adaptive Verification (AV), according to a vLLM blog post by Giancarlo Delfin of Inferact.

The estimator is a logistic regression on the log-odds of the drafter's top token, with coefficients refit from the target model's accept/reject decisions during serving. The feature is enabled with the enable_adaptive_verification option and landed in PR #52228.

On DeepSeek-V4-Flash with DSpark, the blog post reports the estimator beats the trained confidence head on AUC and log-loss and roughly matches it on calibration after excluding the first draft position. Delfin says any existing draft checkpoint gets AV without retraining.

A later vLLM post on X says the feature works with MTP, EAGLE, DFlash, and DSpark checkpoints without a head, and is available on main, but only for draft models whose attention backends support variable-length verification.

Written by AI from the articles below · updated Oct 14, 6:10 PM ET

Check the sources:

Developments

2 developments

  1. Oct 14, 5:56 PM ET · 1 article
    vLLM adds online-calibrated adaptive verification for speculative decoding without trained confidence heads
    vLLM: vLLM adds adaptive verification for draft models without confidence heads
  2. Oct 13, 8:00 PM ET · 1 article
    vLLM adds online-fitted acceptance estimator enabling Adaptive Verification for drafters without a confidence head
    vLLM Blog: vLLM adds an online-fitted acceptance estimator for adaptive verification without a confidence head

Article timeline

The articles in this story. Times are ET.

Oct 14
  1. vLLMOfficial
    vLLM adds adaptive verification for draft models without confidence heads

    AIvLLM says adaptive verification now works with MTP, EAGLE, DFlash, and DSpark draft checkpoints without a trained confidence head or retraining. The drafter's top-token probability ranks drafts well but overestimates acceptance, so a small logistic fit trained online from the target's accept/reject decisions corrects the calibration. The feature is available on main behind enable_adaptive_verification for draft models whose attention backends support variable-length verification.

Oct 13
  1. vLLM BlogOfficialPick
    vLLM adds an online-fitted acceptance estimator for adaptive verification without a confidence head

    AIvLLM says its new online-fitted estimator lets draft models without a confidence head use Adaptive Verification. The estimator uses a logistic regression on the log-odds of the drafter's top token, with coefficients refit from the target model's accept/reject decisions during serving. On DeepSeek-V4-Flash with DSpark, the post reports the estimator outperforms the trained confidence head on AUC and log-loss, and roughly matches it on calibration after excluding the first draft position.

Heat trend

Not enough continuous observations to show a trend yet.