Google launches native PyTorch vLLM and SGLang inference engines on TPU
AIGoogle announces native PyTorch backends for vLLM and SGLang on TPU, built on TorchTPU and developed with Inferact and RadixArk. The engines add production features such as prefix caching, KV cache offloading, and disaggregated serving, with KV cache transfers handled by the open-source TPU Sync library.
AIWhy it matters.
The post explains how PyTorch-native vLLM and SGLang engines run on TPU and how TPU Sync moves KV cache, a concrete design for serving agentic workloads.
Source details
Google Developers Blog · developers.googleblog.comPublished · added here
Original title: Announcing New vLLM and SGLang Inference Engines on TPU