# Preprint finds one setting explains much of an AI serving capacity gap

_Published Thursday, November 5, 2026 at 2:28 PM EST · AI, Science · Latest · Tier 2 — Notable_

A preprint comparing the vLLM and SGLang language-model serving engines finds that matching one configuration setting removes 78% of their capacity gap where that gap is largest. The setting controls whether engines process long prompts in chunks before generating an answer.

Researchers used a reproducible Kubernetes testbed and a generated multi-agent workload. Matching the setting also removed 30 to 67% of the latency gap. SGLang retained a 2.4 to 6.4x advantage in tail latency at equal throughput, consistent with ordering queued requests to reuse cached prompt data.

## Sources

- [cs.LG updates on arXiv.org](https://arxiv.org/abs/2611.05991)

---
Canonical: https://techandbusiness.org/newswire/a3cCvULGw-Jw-UvRq0SVOx
Published: 2026-11-05T19:28:04.027Z
Story chronology: 2026-11-05T05:00:00.000Z
Retrieved: 2026-11-06T22:57:22.390Z
Publisher: Tech & Business (techandbusiness.org)