# HAServe preprint reports higher throughput from coordinated scheduling and memory

_Published Friday, October 16, 2026 at 7:57 AM EDT · AI, Science · Latest · Tier 2 — Notable_

Researchers report that HAServe, a method for coordinating request scheduling and memory retention in language model services, increased throughput by up to 1.08-1.44× across evaluated workloads. The preprint also reports cache hit rates up to 20.59 percentage points higher than the strongest baseline for each metric and setting.

HAServe jointly chooses which requests to admit and which cached intermediate results to retain, using visible demand to minimize predicted latency. Researchers implemented it on SGLang and tested Qwen3-4B-Instruct-2507 and Llama2-13B across five workloads using eight H100 GPUs. The reported gains apply to those evaluated models, workloads and hardware.

## Sources

- [cs.LG updates on arXiv.org](https://arxiv.org/abs/2605.08581)

---
Canonical: https://techandbusiness.org/newswire/37oH09iLDM658G9VLdvk1n
Published: 2026-10-16T11:57:51.108Z
Story chronology: 2026-10-16T04:00:00.000Z
Retrieved: 2026-10-16T15:27:34.403Z
Publisher: Tech & Business (techandbusiness.org)