Google releases TPU Sync, an open-source KV cache transfer library for Cloud TPUs

Google Cloud has released TPU Sync, an Apache-2.0 C++ library that moves KV caches and model weights across TPU HBM, pinned host DRAM, shared storage and the data center network. In internal July 2026 tests on TPU v7x, Google reports over 90% lower KV transfer overhead, with Qwen3 235B median KV transfer latency dropping from several seconds to about 230 ms and average TTFT falling 60% for 235B and 90% for 480B at low concurrency. The library integrates with PyTorch via TorchTPU and with JAX, serving as the KV connector in vLLM-TorchTPU and the P/D transfer backend in SGLang-TorchTPU, though the README says it is not yet recommended for general use.