DeepSeek open-sources DeepInfer, an inference engine for DeepSeek-V4.1-Flash
AIDeepSeek has released DeepInfer on GitHub, the inference engine behind DeepSeek-V4.1-Flash, covering MoE expert parallelism, asynchronous batch scheduling, custom kernels, and speculative decoding. The source says a single cluster processes about 155 trillion tokens per day, and only offline inference is open, while online KV Cache storage and cross-replica load balancing are not yet fully released.
AIWhy it matters.
The article walks through DeepInfer's asynchronous scheduling, state correction, and multi-GPU coordination, giving engineers a concrete view of how large-scale inference handles them.
Loading source posts…
Source details
Leiphone · leiphone.comPublished · added here
Original title: DeepSeek 又一开源重器！深挖 DeepInfer 究竟藏着多少技术狠活？