Google explains how to integrate TorchTPU into TorchTitan and optimize models for TPU
AIGoogle Developers describes running PyTorch training on Cloud TPUs through TorchTitan, using a native torch.device("tpu") backend and a fork of Meta's TorchTitan. The post lays out a model optimization playbook and reports results for Llama 3 405B (about 57% MFU on up to 1,024 chips), Qwen3 235B (32.42% MFU on a 1,024-chip slice), and FLUX.1, where a step time fell from 11.49 seconds to 2.78 seconds.
AIWhy it matters.
The post walks through concrete optimization steps and measured MFU gains on TPU across several models, showing how each change affects training performance.
Source details
Google Developers Blog · developers.googleblog.comPublished · added here
Original title: Integrating TorchTPU into TorchTitan and Optimizing Models for TPU