Baseten speeds up Qwen3-ASR batch transcription sevenfold over vanilla vLLM
AIBaseten says its optimized Qwen3-ASR 1.7B stack transcribes up to 7x more audio per second than out-of-the-box vLLM at 64 concurrent requests on a single RTX PRO 6000. The gains come from native audio decoding, FP8 KV cache with FlashInfer, and removing host-to-device syncs, which raised GPU utilization from 86% to 97%.
AIWhy it matters.
The post names the specific bottlenecks it removed, such as CPU audio decoding and GPU sync stalls, which makes the throughput and cost figures easier to evaluate for batch transcription.
Source details
Baseten Blog · baseten.coPublished · added here
Original title: The fastest, lowest-cost batch transcription with Qwen3-ASR