DeepSeek's open-sourced engine hits 1.7M tok/s 🐳
@deepseek_ai released DeepInfer, which serves V4.1 Flash in production, with 325 tok/s for one chat and about 1.7M tok/s across 64 Nvidia Blackwell GPUs
Run AI models locally -> http://atomic.chat
DeepSeek open-sources DeepInfer inference framework, reaching 1.7M tokens per second
AIDeepSeek has open-sourced DeepInfer, its in-house inference framework, which serves DeepSeek-V4.1-Flash in production. According to atomic.chat's repost of DeepSeek's announcement, it reaches 325 tokens per second for a single chat and about 1.7M tokens per second across 64 Nvidia Blackwell GPUs.
atomic.chatVerified on X
@atomic_chat_hq
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 http://github.com/deepseek-ai/DeepInfer 1/4View quoted post on X
Source details
atomic.chat · x.comPublished · added here
Original title: DeepSeek's open-sourced engine hits 1.7M tok/s 🐳