# Preprint reports faster AI serving after redesigning remote cache loading

_Published Friday, October 16, 2026 at 10:12 AM EDT · Science, AI · Latest · Tier 2 — Notable_

Researchers report that redesigning a remote store for reusable model calculations cut p99 latency from 83.32 seconds to 0.67 seconds on a request stream dominated by cache hits. The preprint studied GLM-5.3 on 8xB300 nodes, where loading cached calculations could take longer than recomputing them.

The changes split storage across processing ranks, loaded data asynchronously and fixed three client bugs. The sharded store increased prefill throughput by 14%, or 34% with an upstream kernel fix. All changes have been submitted to LMCache. The results concern this deployment and request stream; a fixed loading cost still limited the benefit for shorter prompts.

## Sources

- [cs.LG updates on arXiv.org](https://arxiv.org/abs/2610.21500)

---
Canonical: https://techandbusiness.org/newswire/1hQcj6pSjVYclmSJhXmkS7
Published: 2026-10-16T14:12:11.112Z
Story chronology: 2026-10-16T04:00:00.000Z
Retrieved: 2026-10-16T17:01:40.840Z
Publisher: Tech & Business (techandbusiness.org)