# NVIDIA reports faster hybrid-model inference with ReplaySSM

_Published Monday, October 12, 2026 at 12:19 PM EDT · AI, Infrastructure · Latest · Tier 2 — Notable_

![A cube assembled from smaller dark blocks, with glowing cyan, pink, yellow, and green sections against a black background. — Primary](https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background.webp)

NVIDIA says ReplaySSM, developed with Princeton's Dao AI Lab, increases throughput for hybrid AI models and is integrated into TensorRT-LLM, vLLM and FlashInfer. The method also runs in production in GLM-5.3-Flash and Kimi-K3.

Instead of rewriting a model's full fixed-size state after every token, ReplaySSM stores a checkpoint and recent inputs. This reduces memory traffic and lets speculative decoding discard rejected draft tokens without restoring full state snapshots.

At batch size 256, NVIDIA measured decode steps running 1.20× to 1.47× faster across four models. Speculative throughput reached 1.85× and 2.10× vLLM's throughput on two models at batch size 512. Tests at batch size 1 showed no speedup.

## Sources

- [NVIDIA Technical Blog](https://developer.nvidia.com/blog/replayssm-rethinking-state-storage-for-faster-hybrid-model-decoding/)

---
Canonical: https://techandbusiness.org/newswire/aWHhEmPqo4jLmFfPaRVpxs
Published: 2026-10-12T16:19:37.816Z
Story chronology: 2026-10-12T16:00:54.000Z
Retrieved: 2026-10-12T18:43:54.038Z
Publisher: Tech & Business (techandbusiness.org)