Welcome SGLang-TorchTPU to the family!
We're excited to offer a new path for developers to use familiar tools and stacks to support models on TPU, and serve them for agentic workloads!
Check out our write-up on how SGLang-TorchTPU works as a plugin for SGLang, its feature and model coverage, and our early exploration of training on TPU!
Today we launch SGLang-TorchTPU, built with @Google TPU team and Meta @PyTorch team. We’re excited to see it powering production inference at Meta! SGLang-TorchTPU is a TPU plugin for SGLang that brings: - Top open models on TPU: Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, MiniMax H3 - SGLang’s production features for large agentic workloads - Custom MegaKernel: Muse Spark Open hits 770 tok/s decode at bs=1 on InferenceX 8K/1K SGLang-TorchTPU is available now: http://github.com/sgl-project/sglang-torchtpu
This story is featured in top picks: “SGLang launches SGLang-TorchTPU, a TPU plugin for serving open models”
LMSYS Org · x.comPublished · added here
Original title: Welcome SGLang-TorchTPU to the family!
