SGLang-TorchTPU gives people options to choose the hardware that best fits their workloads and economics. We're excited to see many partners already running it in production!
Huge thanks to everyone at Google, Meta, our @sgl_project team, and community who made this happen!
We're committed to helping make TPU a first-class citizen in the SGLang ecosystem and will keep building toward a more flexible, heterogeneous future for AI infrastructure.
Today we launch SGLang-TorchTPU, built with @Google TPU team and Meta @PyTorch team. We’re excited to see it powering production inference at Meta! SGLang-TorchTPU is a TPU plugin for SGLang that brings: - Top open models on TPU: Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, MiniMax H3 - SGLang’s production features for large agentic workloads - Custom MegaKernel: Muse Spark Open hits 770 tok/s decode at bs=1 on InferenceX 8K/1K SGLang-TorchTPU is available now: http://github.com/sgl-project/sglang-torchtpu
This story is featured in top picks: “SGLang launches SGLang-TorchTPU, a TPU plugin for serving open models”
RadixArk · x.comPublished · added here
Original title: SGLang-TorchTPU gives people options to choose the hardware that best fits their workloads and economics. We're excited to see many partn...
