SGLang adds Kimi K3.1 Pro support across NVIDIA, AMD, TPU, Trainium and Zhenwu chips

AISGLang says users can run Kimi K3.1 Pro on NVIDIA Vera Rubin, B300, GB300, B200, GB200, H200 and H100, AMD MI355X and MI350X, Google TPU7x, AWS Trainium 2 and 3, and Zhenwu PPU M890 and 810E. It calls this day-0 support for Trainium, and says the SGLang Trainium code will be available soon. Recipes are provided in the SGLang cookbook.

SGLangOfficial

@sgl_project

View on X

Part of a thread · earlier post

You can run Kimi K3.1 Pro with SGLang on:
· NVIDIA Vera Rubin, B300, GB300, B200, GB200, H200, H100
· AMD MI355X, MI350X
· Google TPU7x
· AWS Trainium 2/3
· Zhenwu PPU M890, 810E

This is the first-ever day-0 support for Trainium! The SGLang Trainium code will be available soon.

Visit our cookbook for recipes: https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K3.1-Pro

Source details

SGLang · x.comPublished Nov 4, 11:26 AM ET

Original title: You can run Kimi K3.1 Pro with SGLang on:

Topics

Deployment & engineering China AI Product updates