7 | Xiaomi
Efficient Long-Context Inference with SGLang: Lessons from Xiaomi MiMo
Chen Zhang & Yingchun Lai share how Xiaomi handles KV caches, disaggregated serving, and scheduling for long-context MiMo workloads.
Xiaomi explains long-context inference serving for MiMo with SGLang
AIXiaomi engineers Chen Zhang and Yingchun Lai describe how the company handles KV caches, disaggregated serving, and scheduling for long-context MiMo workloads using SGLang. The post shares lessons from Xiaomi's production inference setup without listing benchmark figures.
@lmsysorg
Part of a thread · earlier post