Skip to content
Google Developers Blog·· 25 d agoPicksAI score60

Google 将 TorchTitan 接入 TorchTPU,并发布 TPU 模型优化手册

Integrating TorchTPU into TorchTitan and Optimizing Models for TPU

AI summary

Google released a TPU fork of Meta's TorchTitan that runs PyTorch training on Cloud TPUs through TorchTPU's native torch.device("tpu") backend, with TPU-native kernels and a step-by-step optimization playbook. On TPU v7x (Ironwood), Llama 3 405B holds about 57% MFU from 128 to 1,024 chips, Qwen3 235B reaches 32.42% MFU on one 1,024-chip slice, and FLUX.1 reaches 19.4% MFU on 1,024 chips at 93% weak-scaling efficiency. The fork is at https://github.com/google-pytorch/torchtitan/, and a Transformers modeling backend plus an agent onboarding skill extend the same trainer to other checkpoints.

Why it matters

Google 与 Meta 合作把 TorchTitan 移植到 TPU,并给出从 FLUX.1 到 Qwen3 235B 的逐级调优路径与 MFU 数字,可作为 PyTorch 团队评估 TPU 训练可行性的参考。

Source:Google Developers Blog · developers.googleblog.com