# GraphTok preprint reports higher AI serving capacity with GPU tokenization

_Published Monday, October 12, 2026 at 3:06 AM EDT · Science, AI · Latest · Tier 2 — Notable_

Researchers report in a preprint that GraphTok, which converts text into model token IDs on a GPU, increases the traffic one NVIDIA Dynamo frontend can sustain. Within a 2 s P99 target, it handles 518 requests/s of 121k-token traffic, compared with 296 using gigatoken and 18.5 using the default tokenizer.

GraphTok runs the encoding pipeline as one replayable CUDA graph and integrates with unmodified vLLM and NVIDIA Dynamo. It reproduces reference token IDs bit-for-bit on five model families. With a cached prefix in vLLM, it lowers mean time to first token by 2.1-2.4x. Short inputs go to a CPU engine where the GPU cannot win.

## Sources

- [cs.LG updates on arXiv.org](https://arxiv.org/abs/2610.13325)

---
Canonical: https://techandbusiness.org/newswire/I088EG6KaRkLlZsw-32htV
Published: 2026-10-12T07:06:13.171Z
Story chronology: 2026-10-12T04:00:00.000Z
Retrieved: 2026-10-13T11:06:04.577Z
Publisher: Tech & Business (techandbusiness.org)