Google releases TPU model design and optimization guide covering shape rules, LatentMoE and parallelism
TPU Model Design Considerations and Optimization Guide
Google launched the open-source AI-Hypercomputer/tpu-codesign guide, which sets out eight principles for designing models around TPU architecture, from 256-aligned dimensions and static shapes to LatentMoE routing and mapping parallel axes onto physical links. Measured Ironwood results include a 2.8x cut in on-device permute time and +14% training throughput from an l=768 LatentMoE on a 30B/128-expert model, and +57% tok/s from halving expert count while doubling d_ff on a 63B LatentMoE. The guide also covers TPU 8t and TPU 8i features such as native FP4, Boardfly topology and the Collectives Acceleration Engine, and recommends validating architecture changes with wall-clock ablations on a tuned stack.
Google's TPU team lays out eight hardware-grounded design rules for aligning model shapes, MoE routing and parallelism with Ironwood and eighth-generation TPUs, backed by measured throughput gains.
Source:Google Developers Blog · developers.googleblog.com