The post lists five vLLM configuration changes with throughput measured after each step, showing how each contributes to the gain on one GPU.
We nearly doubled throughput for Nemotron 3.5 Super VL on a single GB300 GPU, from 6,058 → 11,840 tok/s, with five configuration changes in vLLM.
These let the model draft tokens ahead, keep the GPU from ever sitting idle, and store the model’s memory at half precision.
Cline · x.comPublished
Original title: We nearly doubled throughput for Nemotron 3.5 Super VL on a single GB300 GPU, from 6,058 → 11,840 tok/s, with five configuration changes ...
