Skip to content
Trending storyDeveloping

vLLM releases v0.33.0 with new models, faster speculative decoding and PyTorch 2.14

1 article1 sourcesince Nov 3Last article 1h ago ·

Overview

AISummary of 1 article

vLLM (@vllm_project) releases v0.33.0, adding support for the models Reflection Beam-1.0, Step-5 Preview, Nemotron Parse 3.5 and MiniCPM-OCR 1.0.

The project reports 1.5–2x NVIDIA GB300 decode latency gains and 39–48% lower time to first token. It also reports a 1.04–1.31x speedup from FLy lossy verify, a speculative decoding method.

The release moves to PyTorch 2.14 and ships CUDA 13.0-only wheels, so users on older CUDA versions cannot use the new wheels. It also adds a new AWS EFA image. These performance figures are vLLM's own claims from its release announcement.

Written by AI from the articles below · updated Nov 3, 7:02 PM ET

Check the sources:

Latest developmentNov 3, 6:53 PM ET
vLLM v0.33.0 adds new models, faster speculative decoding, and PyTorch 2.14 support

Article timeline

The articles in this story. Times are ET.

Nov 3
  1. vLLMOfficial
    vLLM v0.33.0 adds new models, faster speculative decoding, and PyTorch 2.14 support

    AIvLLM v0.33.0 adds support for new models including Reflection Beam-1.0, Step-5 Preview, Nemotron Parse 3.5, and MiniCPM-OCR 1.0. The release reports 1.5–2x NVIDIA GB300 decode latency gains, 39–48% lower TTFT, and a 1.04–1.31x speedup from FLy lossy verify. It also moves to PyTorch 2.14 with CUDA 13.0-only wheels and ships a new AWS EFA image.

    Image from @vllm_project's post

Heat Trend

Not enough continuous observations to show a trend yet.