TreeWY preprint reports lower memory use for hybrid model serving

Researchers report that TreeWY reduces memory use when serving hybrid language models, improving throughput and time to first token when memory limits performance. The preprint tests Qwen3.5 35B and 397B models. Speculative decoding checks several proposed tokens together. Existing systems save a model's recurrent state at every draft position; TreeWY replaces those snapshots with a smaller matrix and reconstructs only the accepted state. The benchmarks retain essentially identical acceptance length, but the method costs a few percent where memory does not constrain serving. Wider draft trees become feasible within the same memory budget, though they do not yet improve throughput.