# TreeWY preprint reports lower memory use for hybrid model serving

_Published Monday, October 26, 2026 at 1:30 PM EDT · AI, Science · Latest · Tier 2 — Notable_

Researchers report that TreeWY reduces memory use when serving hybrid language models, improving throughput and time to first token when memory limits performance. The preprint tests Qwen3.5 35B and 397B models.

Speculative decoding checks several proposed tokens together. Existing systems save a model's recurrent state at every draft position; TreeWY replaces those snapshots with a smaller matrix and reconstructs only the accepted state.

The benchmarks retain essentially identical acceptance length, but the method costs a few percent where memory does not constrain serving. Wider draft trees become feasible within the same memory budget, though they do not yet improve throughput.

## Sources

- [cs.AI updates on arXiv.org](https://arxiv.org/abs/2608.20961)

---
Canonical: https://techandbusiness.org/newswire/u88hWrx-VZ7P9MkA9dhv0x
Published: 2026-10-26T17:30:41.839Z
Story chronology: 2026-10-26T04:00:00.000Z
Retrieved: 2026-10-26T20:02:01.094Z
Publisher: Tech & Business (techandbusiness.org)