We collaborated on the model bringup with @sgl_project, validating checkpoint accuracy and optimizing the serving path to support Kimi K3.1 Pro on Day 0.
https://x.com/sgl_project/status/2118017145548451860?a=j6Hf-0G7ckk80aoguqo&s=20
Modal and SGLang support Kimi K3.1 Pro serving on day one
AIModal says it collaborated with SGLang on the model bringup for Kimi K3.1 Pro, validating checkpoint accuracy and optimizing the serving path for Day 0 support. SGLang says Kimi K3.1 Pro replaces dense MLA with sparse attention (DSA), which it calls more efficient and easier to serve.
@modal
Part of a thread · earlier post
Kimi K3.1 Pro replaces dense MLA with sparse attention (DSA). It's much more efficient and easier to serve. SGLang delivers on Day 0: · Up to 33.7K tok/s per GPU (AgentX, GB300) · 450 tok/s single user with DFlash2 + MegaKernel · NVIDIA Vera Rubin / Blackwell / Hopper, AMD, TPU, Trainium, PPU
Source details
Modal · x.comPublished
Original title: We collaborated on the model bringup with @sgl_project, validating checkpoint accuracy and optimizing the serving path to support Kimi K3...