Trending storyDeveloping
Xiaomi shares long-context MiMo inference lessons using SGLang
2 articles1 sourcesince Nov 4Last article 1h ago ·
Article timeline
The articles in this story. Times are ET.
Nov 4
LMSYS OrgOfficialXiaomi shares lessons on long-context MiMo inference with SGLangAIXiaomi engineers Chen Zhang and Yingchun Lai describe how the company handles KV caches, disaggregated serving, and scheduling for long-context MiMo workloads using SGLang. The post presents these approaches as lessons from Xiaomi's MiMo deployments.
- LMSYS OrgOfficialXiaomi explains long-context inference serving for MiMo with SGLang
AIXiaomi engineers Chen Zhang and Yingchun Lai describe how the company handles KV caches, disaggregated serving, and scheduling for long-context MiMo workloads using SGLang. The post shares lessons from Xiaomi's production inference setup without listing benchmark figures.
