GraphTok preprint reports higher AI serving capacity with GPU tokenization

Researchers report in a preprint that GraphTok, which converts text into model token IDs on a GPU, increases the traffic one NVIDIA Dynamo frontend can sustain. Within a 2 s P99 target, it handles 518 requests/s of 121k-token traffic, compared with 296 using gigatoken and 18.5 using the default tokenizer. GraphTok runs the encoding pipeline as one replayable CUDA graph and integrates with unmodified vLLM and NVIDIA Dynamo. It reproduces reference token IDs bit-for-bit on five model families. With a cached prefix in vLLM, it lowers mean time to first token by 2.1-2.4x. Short inputs go to a CPU engine where the GPU cannot win.