Google releases TPU Sync open-source KV cache library
Start here
Google Cloud has released TPU Sync, an Apache-2.0 C++ library that moves KV caches and model weights across TPU HBM, pinned host DRAM, shared storage and the data center network. In internal July 2026 tests on TPU v7x, Google reports over 90% lower KV transfer overhead, with Qwen3 235B median KV transfer latency dropping from several seconds to about 230 ms and average TTFT falling 60% for 235B and 90% for 480B at low concurrency. The library integrates with PyTorch via TorchTPU and with JAX, serving as the KV connector in vLLM-TorchTPU and the P/D transfer backend in SGLang-TorchTPU. Google's README says it is not yet recommended for general use; storage relies on experimental POSIX backends including local SSDs and Google Cloud Managed Lustre, and RL and storage tiering are experimental. All performance figures are self-reported by Google.
Generated from reports · updated 1 h ago
Report timeline
Follow the reports to see the event from different sides.
- MarkTechPostGoogle releases TPU Sync, an open-source KV cache transfer library for Cloud TPUs
Google Cloud has released TPU Sync, an Apache-2.0 C++ library that moves KV caches and model weights across TPU HBM, pinned host DRAM, shared storage and the data center network. In internal July 2026 tests on TPU v7x, Google reports over 90% lower KV transfer overhead, with Qwen3 235B median KV transfer latency dropping from several seconds to about 230 ms and average TTFT falling 60% for 235B and 90% for 480B at low concurrency. The library integrates with PyTorch via TorchTPU and with JAX, serving as the KV connector in vLLM-TorchTPU and the P/D transfer backend in SGLang-TorchTPU, though the README says it is not yet recommended for general use.
Heat for this event
Not enough continuous observations yet to draw a trend.