Baseten reports 7x faster Qwen3-ASR 1.7B batch transcription over vanilla vLLM
Overview
Baseten says its optimized Qwen3-ASR 1.7B stack transcribes up to 7x more audio per second than out-of-the-box vLLM at 64 concurrent requests on a single RTX PRO 6000.
The gains come from native audio decoding, FP8 KV cache with FlashInfer, and removing host-to-device syncs, which Baseten says raised GPU utilization from 86% to 97%.
Baseten puts the cost at $0.0013 per audio hour, compared with $0.011 on vanilla vLLM. The company says the optimized deployment is the default in its Model Library, can be deployed in one click, and works with any OpenAI-compatible transcription client. It can also be paired with Nemotron 3 Diarization for speaker-attributed transcripts. These figures are Baseten's own, from its first-party blog post.
Written by AI from the articles below · updated Oct 28, 2:05 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- Baseten BlogOfficialPickBaseten speeds up Qwen3-ASR batch transcription sevenfold over vanilla vLLM
AIBaseten says its optimized Qwen3-ASR 1.7B stack transcribes up to 7x more audio per second than out-of-the-box vLLM at 64 concurrent requests on a single RTX PRO 6000. The gains come from native audio decoding, FP8 KV cache with FlashInfer, and removing host-to-device syncs, which raised GPU utilization from 86% to 97%. Baseten puts the cost at $0.0013 per audio hour, versus $0.011 on vanilla vLLM, and the optimized deployment is available in its Model Library.
Heat Trend
Not enough continuous observations to show a trend yet.