Skip to content
View original post on X: Tianyi CuiX· 65/100AI score65/100

DeepSeek open-sources DeepInfer, a large-scale inference framework under Apache-2.0

AISummary

DeepSeek has open-sourced DeepInfer, a high-performance, large-scale inference framework, under the Apache-2.0 license, according to Tianyi Cui's post. The post lists optimized kernels, attention data parallelism, MoE expert parallelism with EPLB, asynchronous scheduling, batch-invariant inference, deterministic sampling, and structured output as supported features. The repository is at

Post on XView on X
Tianyi CuiVerified on X
@tianyi

DeepSeek just open sourced its high-performance, large-scale inference framework DeepInfer in Apache-2.0 license.

https://github.com/deepseek-ai/DeepInfer

Supported features:

  • Optimized kernels, model and sampler runtime
  • Parallelism: Attention DP / balanced CP, MoE EP + EPLB
  • DSpark and confidence-scheduled verification
  • SWA bounded replay
  • Asynchronous scheduling (zero-overhead batch scheduler)
  • Batch-invariant inference
  • Deterministic sampling
  • Structured output

Source: Tianyi Cui · x.comPublished · added here

Topics

DeepSeekOpen-source ecosystemDeployment & engineeringChina AI