# AWS reports faster vLLM responses with push-based cache transfers

_Published Monday, October 19, 2026 at 12:18 PM EDT · AI · Latest · Tier 2 — Notable_

![AWS reports faster vLLM responses with push-based cache transfers — Primary](https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/15/ML-21812-featured-image.png)

AWS detailed a vLLM feature that it says improves response latency by pushing cached prompt calculations between servers. In its Qwen3-32B benchmark on EC2 P5en instances connected with EFA, push mode delivered up to 3x better time to first token and up to 30 percent better time per output token than pull mode.

The connector registers destination memory while prompt processing continues and prepares transfers on a background thread, reducing work that delays generation. The benchmark disabled prefix caching. Push mode also requires earlier selection of the generation server; pull mode retains more flexibility for load-aware routing and supports bidirectional transfers.

## Sources

- [AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/push-based-kv-transfer-to-improve-ttft-and-tpot-for-disaggregated-inference-in-vllm/)

---
Canonical: https://techandbusiness.org/newswire/DgnKdf3Da13ePybt51xaOR
Published: 2026-10-19T16:18:27.102Z
Story chronology: 2026-10-19T16:05:17.000Z
Retrieved: 2026-10-19T19:17:32.443Z
Publisher: Tech & Business (techandbusiness.org)