AWS reports faster vLLM responses with push-based cache transfers

AWS detailed a vLLM feature that it says improves response latency by pushing cached prompt calculations between servers. In its Qwen3-32B benchmark on EC2 P5en instances connected with EFA, push mode delivered up to 3x better time to first token and up to 30 percent better time per output token than pull mode. The connector registers destination memory while prompt processing continues and prepares transfers on a background thread, reducing work that delays generation. The benchmark disabled prefix caching. Push mode also requires earlier selection of the generation server; pull mode retains more flexibility for load-aware routing and supports bidirectional transfers.