SenseTime's Large Device platform serves 6 trillion tokens a day with heterogeneous inference
AISenseTime's Large Device platform, led by CTO Xuan Shanming, now serves 6 trillion tokens per day, and plans 10 trillion per day by Q4 2026. Its heterogeneous inference cluster splits Prefill onto domestic chips and Decode onto high-bandwidth GPUs, and says that setup raises total token throughput to 5 times the baseline on the same number of high-end cards in a specific test configuration.
Loading source posts…
Source details
QbitAI · 量子位的朋友们 · qbitai.comPublished · added here
Original title: 日均6万亿Token服务背后，商汤大装置如何提升推理效能？