DeepSeek just open-sourced DeepInfer, the inference engine behind its production services, under the Apache 2.0 license.
and it got 6.3K+ stars in 8 hours
The release runs only DeepSeek-V4.1-Flash, on NVIDIA SM100 GPUs or Huawei Ascend 950 chips, and covers offline inference rather than a complete serving stack.
It omits the parts that turn an engine into a service: a KV cache system, prefill and decode disaggregation, and load balancing across replicas.
What ships is the speed layer, including DSpark, which drafts up to 5 tokens ahead and uses confidence estimates to schedule their verification.
Bounded replay trims prefill by recomputing only a 140-token suffix sized to cover the sliding attention window.
A companion library, deepseek-recipe, will turn Chat Completions, Responses and Messages requests into DeepSeek V4 and V4.1 prompts that match the official API’s behaviour.