Google releases TPU Sync to move AI context between accelerator and host memory

Google released TPU Sync, an open-source library that moves stored AI context and model weights across TPU accelerator memory, host memory and networks while computation continues. Available with Python packages for PyTorch and JAX, it integrates with vLLM-TorchTPU, SGLang-TorchTPU and llm-d on Google Kubernetes Engine. The library lets serving systems reuse previously processed prompts and transfer their context between machines that process prompts and generate responses. Google reports 30% to 70% higher token output throughput for a 235B-parameter model during sustained traffic spikes, depending on traffic distribution. Those results come from internal testing under controlled laboratory conditions. Actual gains depend on workload, configuration and network bandwidth; storage tiering and reinforcement-learning transfer modules remain experimental.