DeepSeek open-sources DeepInfer, its inference framework for DeepSeek-V4.1-Flash
Article timeline
The articles in this story. Times are ET.
SGLang@sgl_projectOfficialPickDeepSeek open-sources DeepInfer, its in-house inference frameworkAIDeepSeek has open-sourced DeepInfer, its in-house inference framework, which it says powers DeepSeek-V4.1-Flash at scale with high throughput and low latency. SGLang congratulates DeepSeek and says it is excited about the large-batch and multi-GPU inference work and plans to bring some of those ideas into SGLang.
Orange AI@oran_geXDeepSeek open-sources DeepInfer, its in-house inference frameworkAIDeepSeek has released DeepInfer, its in-house inference framework, as open source on GitHub. According to the quoted DeepSeek post, the framework powers DeepSeek-V4.1-Flash at scale with high throughput and low latency, and the company is sharing its production stack to help lower compute costs and energy use. The author says the move will push down model prices across the industry.
elie@eliebakouchXDeepSeek open-sources DeepInfer, its production inference stack for V4.1AIDeepSeek has open-sourced DeepInfer, the in-house inference framework it uses to run DeepSeek-V4.1-Flash at scale with high throughput and low latency. The company says it is sharing the production stack to help the community lower compute costs and energy use.
vLLM@vllm_projectOfficialDeepSeek open-sources DeepInfer inference framework built primarily on vLLMAIvLLM thanks DeepSeek for open-sourcing DeepInfer under Apache 2.0, noting it was built primarily on vLLM. DeepSeek says DeepInfer powers DeepSeek-V4.1-Flash at scale and shares its production stack to lower compute costs. vLLM says it will keep improving DeepSeek V4.1 Flash inference efficiency using ideas from DeepInfer.
Dongxi NLP@dongxi_nlpXDeepSeek open-sources DeepInfer, a large-scale inference frameworkAIDongxi NLP reports that DeepSeek has open-sourced DeepInfer, a high-performance, large-scale inference framework, under the Apache-2.0 license. The quoted post lists supported features including optimized kernels, Attention DP and MoE EP parallelism, asynchronous scheduling, batch-invariant inference, deterministic sampling, and structured output. The repository is at
meng shao@shao__mengXPickDeepSeek open-sources DeepInfer, its offline high-performance inference frameworkAIDeepSeek has open-sourced DeepInfer, an offline inference framework under Apache 2.0 that it says is the same stack serving its production services. The framework integrates FlashMLA, DeepGEMM, DeepEP and EPLB kernels and supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. The post highlights confidence-based draft-token verification, bounded YoCo SWA replay, and batch-invariant deterministic output, which currently works only on NVIDIA.
Guizang@op7418XDeepSeek open-sources its in-house DeepInfer inference frameworkAIDeepSeek has released DeepInfer, its in-house inference framework, as open source on GitHub. The company says it runs DeepSeek-V4.1-Flash at scale with high throughput and low latency, and shares its production stack to help lower compute costs and energy use. The author, Guizang, says the framework can deliver throughput and latency similar to DeepSeek's own.
DeepSeek@deepseek_aiOfficialDeepSeek open-sources DeepInfer, its inference framework for DeepSeek-V4.1-FlashAIDeepSeek releases DeepInfer, its in-house inference framework, as open source on GitHub. The company says the framework powers DeepSeek-V4.1-Flash at scale with high throughput and low latency. DeepSeek is sharing its production stack to help lower compute costs and energy use.
Tianyi Cui@tianyiXDeepSeek open-sources DeepInfer, a large-scale inference framework under Apache-2.0AIDeepSeek has open-sourced DeepInfer, a high-performance, large-scale inference framework, under the Apache-2.0 license, according to Tianyi Cui's post. The post lists optimized kernels, attention data parallelism, MoE expert parallelism with EPLB, asynchronous scheduling, batch-invariant inference, deterministic sampling, and structured output as supported features. The repository is at