Cline reports 11,840 tokens/s for Nemotron 3.5 Super VL on one GB300 GPU

Cline says five vLLM configuration changes raised Nemotron 3.5 Super VL throughput on a single GB300 GPU from 6,058 to 11,840 output tokens per second.

The changes let the model draft tokens ahead, keep the GPU from sitting idle, and store the model's memory at half precision. In its chart, Cline lists the steps: MTP decoding about 7,000 tokens/s, async with FlashInfer 7,800, changing MTP 3 to MTP 1 9,500, and FP16 state 11,840 at 256 concurrent requests.

Cline published a blog post on the optimizations and on the future of local models and agentic inference. The figures are Cline's own measurements as posted on X.