RadixArk explains how it optimized Muse Spark Open on TPU v7
AIRadixArk explains inference optimizations that serve Meta's 816B-parameter Muse Spark Open model on a single four-chip TPU v7 host using SGLang-TorchTPU. The main change is a megakernel that runs all 62 layers of a decode or verify step as one program, which the post says halves time per output token for one or two concurrent requests. The post also reports gains from attention pipelining, KV cache layout, and sequence-parallel prefill, and says the code is open source in the SGLang-TorchTPU repository.
AIWhy it matters.
The post walks through each optimization on TPU v7 with measured before-and-after timings, showing how a megakernel and pipelining changed decode, attention, and prefill costs.
Source details
PyTorch Blog · Haolin Fu, Xiaoyu Zhang, Junyan Xu, Wangyuan Zhang, Haoguang Cai, RadixArk Team · pytorch.orgPublished · added here
Original title: Optimizing Muse Spark Open on TPU v7 with SGLang-TorchTPU