Google 将 TorchTitan 接入 TorchTPU,并发布 TPU 模型优化手册
Integrating TorchTPU into TorchTitan and Optimizing Models for TPU
เนื้อหานี้ยังไม่มีในภาษาของคุณ จึงแสดงเป็นEnglish
Google released a TPU fork of Meta's TorchTitan that runs PyTorch training on Cloud TPUs through TorchTPU's native torch.device("tpu") backend, with TPU-native kernels and a step-by-step optimization playbook. On TPU v7x (Ironwood), Llama 3 405B holds about 57% MFU from 128 to 1,024 chips, Qwen3 235B reaches 32.42% MFU on one 1,024-chip slice, and FLUX.1 reaches 19.4% MFU on 1,024 chips at 93% weak-scaling efficiency. The fork is at https://github.com/google-pytorch/torchtitan/, and a Transformers modeling backend plus an agent onboarding skill extend the same trainer to other checkpoints.
Google 与 Meta 合作把 TorchTitan 移植到 TPU,并给出从 FLUX.1 到 Qwen3 235B 的逐级调优路径与 MFU 数字,可作为 PyTorch 团队评估 TPU 训练可行性的参考。
แหล่งข่าว:Google Developers Blog · developers.googleblog.com