Baseten optimized Qwen3-ASR to achieve low-cost batch speech transcription
The story so far
Baseten has achieved low-cost, high-throughput batch speech transcription by optimizing Qwen3-ASR 1.7B on the Baseten platform. Its vLLM stack supports 7 times the throughput of standard vLLM on an RTX PRO 6000, with a maximum of 3,128 times the real-time factor, enhancing performance through native audio decoding, FP8 KV cache quantization, and reducing host-device synchronization.
AI-written from the reports · Updated 21 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Baseten BlogThe fastest, lowest-cost batch transcription with Qwen3-ASR
Baseten achieved the lowest cost per minute for batch speech transcription by optimizing Qwen3-ASR 1.7B (175B parameters, 5.4% WER) on Baseten. Its vLLM stack supports 7 times the throughput of standard vLLM on RTX PRO 6000, with a maximum real-time factor of 3,128, enhanced by native audio decoding, FP8 KV cache quantization, and reduced host-device synchronization.