AWS details vLLM cache reuse that cuts multi-turn response delays

AWS reported that bidirectional cache transfers in vLLM reduced time to first token by up to 1.6x for Qwen3-32B and up to 3.3x for GLM-5-FP8 in multi-turn benchmarks. Tests used separate prompt-processing and generation instances on EC2 P5en hardware connected with EFA, with prompt-side prefix caching disabled. The feature returns cached calculations from generated responses to the prompt-processing server on later turns, avoiding repeated computation. Deployment requires vLLM v0.21.0 or later and a router that tracks cache locations. Generated caches remain available for a configurable period, defaulting to 480 seconds; longer retention consumes accelerator memory.