7 | @XiaomiMiMo
Efficient Long-Context Inference with SGLang: Lessons from Xiaomi MiMo
Chen Zhang & Yingchun Lai share how Xiaomi handles KV caches, disaggregated serving, and scheduling for long-context MiMo workloads.
Xiaomi shares lessons on long-context MiMo inference with SGLang
AIXiaomi engineers Chen Zhang and Yingchun Lai describe how the company handles KV caches, disaggregated serving, and scheduling for long-context MiMo workloads using SGLang. The post presents these approaches as lessons from Xiaomi's MiMo deployments.
@lmsysorg
Part of a thread · earlier post