DeepSeek open-sources DeepInfer, its production inference engine, with claimed cache-heavy serving economics
AIDeepSeek released DeepInfer, an open-source inference engine behind its V4.1 Flash model, alongside production claims of 155 trillion tokens per day from one cluster and $38 revenue per accelerator-hour at off-peak prices. The source notes that these are first-party figures, that KV-cache management and load-balancing components remain unreleased, and that the throughput depends heavily on a 96% input-cache hit rate.
Source details
Latent Space · latent.spacePublished · added here
Original title: [AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback