Skip to content
View original post on X: SGLangOfficial
Published
Pick72/100

SGLang launches SGLang-TorchTPU, a TPU plugin for serving open models

AISGLang has launched SGLang-TorchTPU, a TPU plugin for SGLang built with Google's TPU team and Meta's PyTorch team. The plugin lists Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, and MiniMax H3 among supported open models.

AIWhy it matters.

The post gives the plugin's supported models and a specific decode speed on InferenceX, letting readers judge what the TPU support covers.

SGLangVerified on XOfficial
@sgl_project
View on X

Today we launch SGLang-TorchTPU, built with @Google TPU team and Meta @PyTorch team.

We’re excited to see it powering production inference at Meta!

SGLang-TorchTPU is a TPU plugin for SGLang that brings:

  • Top open models on TPU: Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, MiniMax H3
  • SGLang’s production features for large agentic workloads
  • Custom MegaKernel: Muse Spark Open hits 770 tok/s decode at bs=1 on InferenceX 8K/1K

SGLang-TorchTPU is available now: http://github.com/sgl-project/sglang-torchtpu

Google Cloud Tech@GoogleCloudTech
Big day for the @PyTorch ecosystem at #PyTorchConf2026 🚀 We’re officially open-sourcing TorchTPU—enabling native, hardware-accelerated PyTorch execution across Cloud TPUs and custom infrastructure. Here’s everything we launched to supercharge your training & inference workloads 🧵👇 https://goo.gle/4Bl0zdG
View quoted post on X
Source details

SGLang · x.comPublished · added here

Original title: Today we launch SGLang-TorchTPU, built with @Google TPU team and Meta @PyTorch team.

Topics

Google / GeminiMeta / LlamaOpen-source ecosystemDeployment & engineeringProduct updates