# AWS details vLLM cache reuse that cuts multi-turn response delays

_Published Monday, October 19, 2026 at 12:18 PM EDT · AI · Latest · Tier 2 — Notable_

![AWS details vLLM cache reuse that cuts multi-turn response delays — Primary](https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/15/ML-21811-featured-image.png)

AWS reported that bidirectional cache transfers in vLLM reduced time to first token by up to 1.6x for Qwen3-32B and up to 3.3x for GLM-5-FP8 in multi-turn benchmarks. Tests used separate prompt-processing and generation instances on EC2 P5en hardware connected with EFA, with prompt-side prefix caching disabled.

The feature returns cached calculations from generated responses to the prompt-processing server on later turns, avoiding repeated computation. Deployment requires vLLM v0.21.0 or later and a router that tracks cache locations. Generated caches remain available for a configurable period, defaulting to 480 seconds; longer retention consumes accelerator memory.

## Sources

- [AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/improve-ttft-for-multi-turn-agentic-workloads-with-bi-directional-kv-transfer-in-vllm/)

---
Canonical: https://techandbusiness.org/newswire/jpqnbZ7WuCxVc6fAsOsV5w
Published: 2026-10-19T16:18:37.519Z
Story chronology: 2026-10-19T16:03:23.000Z
Retrieved: 2026-10-19T19:21:32.745Z
Publisher: Tech & Business (techandbusiness.org)